The navigation ability and the perception of the objects in the environment are the basis for the mobile robots to explore an unknown environment. Understanding the surroundings and determining the objects in the environment of one of the major tasks in this regard. For this purpose, we propose a method with RGB-D sensor-based material mapping with simultaneous localization and mapping (SLAM) fusion. This method integrates the material classification network and SLAM to obtain the semantic map. This enables the robot to explore an unknown environment autonomously and identify the objects and materials in the environment. The material recognition or classification is done based on the images taken from the RGB-D camera, and SLAM is done using ORB-SLAM2. The RGB and Depth features are used for material classification, and we also explored different fusion methods, late fusion, and complementarity-aware (CA) fusion. Experiments are conducted to demonstrate this mechanism with a mobile robot, and an RGB-D camera installed, exploring an unknown environment consisting of objects of different material types. The results show that the mobile robot can successfully navigate the area by classifying and creating the material map.