Artificial Intelligence is revolutionizing the field of transportation, offering innovative solutions to longstanding traffic and infrastructure challenges. This work presents novel deep-learning (DL) frameworks designed for two related tasks: vehicle classification and traffic accident prediction, where vehicle classes are key inputs for traffic accident prediction. For vehicle classification, the proposed DL framework leverages two types of features extracted from vehicle images: (1) high-level embeddings from state-of-the-art vision transformers (ViTs) and (2) localized vehicle wheel position features obtained through real-time object detection models. This study utilizes transfer learning and self-supervised pretraining to obtain high-quality image embeddings and further employ a novel wheel masking strategy to harmonize the two types of features. The best model achieves a Top-1 classification accuracy of 97.2%, underscoring the framework’s effectiveness in achieving high precision in the vehicle classification task. Building upon the vehicle classification work, Graph Neural Networks (GNN) are further designed for traffic accident prediction, where vehicles are represented as individual nodes in the graph and the vehicle classes serve as key node features. The prediction framework encompasses graph classification (predicting occurrence of a traffic accident) and collision time regression (estimating when the accident will occur). The GNN model achieves 86.1% accuracy in graph classification and an average error of 0.36 seconds in collision time regression. The prediction framework offers new opportunities for enhancing collision warnings and improving safety measures. Both the vehicle classification and traffic accident prediction frameworks demonstrate the potential of DL methods to effectively address complex transportation challenges.