Intrusion is one of the most serious rule-breaking behaviors but it has been rarely seen in present studies since the existing manual safety observation can hardly obtain massive and complex intrusion related information effectively. This paper explores how weather and temperature impact on various kinds of intrusion behaviors on construction site. A location tracking technology combined with BIM (building information modeling) was applied for identifying and recording intrusion behaviors of 147 construction workers for 4 months. The results, empirically tested by ANOVA, show that weather, temperature, and temperature difference all have outstanding effects on both intrusion frequency and duration, however the interactive effects are not statistically significant. The analysis further indicates in good weather or uncomfortable temperature conditions, both intrusion frequency and duration are higher than those in other experiment conditions. Through on-site interviews, the corresponding reasons are found. Then recommendations to control this type of risk-taking behavior are proposed like the application of location tracking technology, timely and targeted safety training, safe layout, and reasonable workload. The findings in this study also provide insights into the use of location tracking technology for intrusion management and safety improvement in the construction industry.