Family 47 glycohydrolases cleave 1,2-mannose linkages on Asn-linked oligomannose structures by an inverting mechanism that involves an substrate interaction with an enzyme-bound Ca2+ ion for substrate distortion into the transition state. Among enzymes in this family, ER -mannosidase I (ERManI) and Golgi -mannosidase IA (GMIA) are key enzymes involved in N-linked glycan biosynthesis in the endoplasmic reticulum (ER) and Golgi complex. The structures of ERManI and Golgi ManIA are similar in overall protein fold, but have complementary activities in their cleavage of the natural Man9GlcNAc2 substrate. ERManI preferentially cleaves a single 1,2-Man residue from the central B-branch of the tri-branched oligomannose substrate, whereas GMIA preferentially cleaves three 1,2Man residues from branches A and C, but poorly cleaves the B branch 1,2-Man residue. In order to determine the structural basis for the substrate specificity differences, human ERManI and murine GMIA were inactivated by replacing their enzyme-bound Ca2+ with La3+, and co-crystallized with the substrate analog Man9GlcNAc2-PA. The crystal structures were resolved at resolutions of 1.65 for ERManI and 1.77 for GMIA. The enzyme-substrate complex structures reveal that GMIA binds the non-reducing terminal 1,2-Man of branch A in the -1 enzyme subsite, while the remainder of the glycan has extended interactions in the enzyme cleft. In contrast, ERManI binds the -1 enzyme subsite with the terminal 1,2-Man of branch B with a distinctive set of interactions between the glycan substrate and the enzyme cleft. Non-conserved residues in the topologically equivalent positions of the glycan binding clefts were swapped between ERManI and GMIA and the mutations led to substrate specificity changes as well as the compromised enzymatic activity and substrate binding affinity in some cases. In order to search for the selective inhibitors towards ERManI as the potential therapeutic agents for protein misfolding disorders, a collection of kifunensine and 1-deoxymannojirimycin derivatives have been screened. Varying degrees of selectivity of inhibition were found among several compounds. The structural and enzymatic studies on ERManI and GMIA in this work provided more insights into the structural basis of substrate specificity for members of this enzyme family.