This research presents a machine learning pipeline for evaluating the realism of synthetic lidar point clouds generated by AWSIM, an open-source simulator developed by TIER IV for Autoware, a leading self-driving system. The lidar realism evaluation enables reliable simulation testing for Autoware's perception algorithms. Using geometric descriptors and XGBoost classification, Tier IV will quantify how closely simulated lidar data resembles real-world sensor data. The company's experiments validated that distance Gaussian noise significantly improves synthetic data realism, particularly on vertical surfaces. The pipeline achieves 91% accuracy and provides actionable feedback through SHAP-based feature importance analysis and realism score visualization.

• Practical methodology: a pipeline approach to quantify lidar simulation realism using geometric descriptors and realism scoring metrics
• How real-world lidar data from Tier IV's actual autonomous driving operations enables reliable simulation realism evaluation
• How simulation validation supports Autoware systems that Tier IV has deployed in real-world services and experiments
• Insights into AWSIM and Autoware simulation tools available for autonomous driving development projects
• Key differences between synthetic and real lidar data and practical approaches to bridge this critical gap