AI Safety Incident Reporting System (Power Platform Project)

Project Explanation & Interview Questions for AI Powered Safety Incident Reporting System

1. What Business Problem Was Solved?

The organization was managing safety incident reporting across 12 offshore platforms and 35 onshore facilities with over 3,200 field workers. However, the entire process was manual and fragmented. Incidents such as injuries, near-misses, equipment failures, and environmental spills were reported through paper forms, emails, and phone calls. This resulted in delays of 2–3 days just to log a single incident into the system. Additionally, there was no standardized approach to root cause analysis, and the company lacked predictive insights to identify high-risk locations or equipment. Regulatory reporting for OSHA and EPA was also manual and error-prone. The biggest concern was the Lost-Time Injury Rate (LTIR), which stood at 2.1 — significantly higher than the industry average of 1.6. This highlighted the need for a complete digital transformation of the safety system.

2. What Was the Solution Approach for AI Powered Safety Incident Reporting System?

To solve this, a fully digital and AI-powered safety platform was designed using Power Platform and Azure AI services. A mobile-first Canvas App was built for field workers with full offline capability. Workers could capture photos, record voice notes, and submit geo-tagged incident reports even without network connectivity. AI played a major role in automating the process. AI Builder detected hazards in images, Azure OpenAI converted voice input into structured data, and text classification identified incident type and severity. Power Automate routed incidents to the correct investigator, while investigators used a Model-Driven App with guided root cause analysis. Azure Machine Learning added predictive capabilities, and Power BI dashboards provided insights.

3. How Does the End-to-End Architecture Work?

The architecture was designed in three logical layers to ensure scalability and reliability. Field Layer: Workers used an offline-first Canvas App to capture incident data using photos, voice, and GPS. Power Platform Layer: Dataverse stored all data, Power Apps handled user interaction, Power Automate managed workflows, and Power BI provided analytics. AI Layer: Speech-to-text processed voice inputs, computer vision analyzed images, GPT structured incident data, and Azure ML predicted high-risk scenarios. This layered architecture ensured that the system was both scalable and intelligent.

4. How Was AI Used in This AI Powered Safety Incident Reporting System Solution?

AI was embedded across the entire workflow, not just as an add-on feature. Speech-to-text converted worker voice input into text even in noisy environments. Azure OpenAI structured the data and suggested investigation steps. AI Builder detected hazards in images like spills or missing PPE. Additionally, text classification categorized incidents automatically, and Azure Machine Learning predicted future risks based on historical data. This transformed the system from reactive reporting to proactive safety management.

5. What Challenges Did You Face?

The project faced multiple real-world challenges, especially due to offshore environments. Connectivity was unreliable due to satellite internet, causing frequent failures. Speech-to-text accuracy was initially low due to noise and accents. Image quality was inconsistent, affecting AI detection. There was also cultural resistance — workers feared blame, leading to under-reporting. Finally, the safety team raised concerns about AI bias across different worker groups.

6. How Did You Solve These Challenges?

Each challenge was addressed with a structured solution. An offline-first architecture was implemented with local storage, sync queues, and retry mechanisms. Speech models were trained using real offshore audio and safety terminology, improving accuracy from 74% to 91%. Photo guidance and AI validation improved image quality. A “Just Culture” approach encouraged reporting by focusing on learning instead of blame. AI fairness testing ensured consistent results across groups, and regulatory compliance was enforced using validation rules and audit-ready structures.

7. What Was the Business Impact?

The solution delivered strong measurable results. Incident reporting increased by 140%, capturing previously missed near-misses. The LTIR dropped from 2.1 to 1.3, representing a 38% improvement in safety. Data entry time reduced from 2–3 days to just 15 minutes, and investigation time dropped from 18 days to 6 days. The predictive model identified 15 high-risk scenarios and helped prevent approximately 8 serious incidents. Overall adoption reached 92% among field workers.

8. Interview Question: How Did You Handle Offline Functionality?

The solution used a full offline-first architecture. All data — including photos and voice notes — was stored locally with compression. A sync queue managed uploads, with auto-sync when connectivity improved and manual sync options for users. Incremental uploads ensured that failed uploads resumed from where they stopped. The system was fully tested in airplane mode to ensure reliability in real offshore conditions.

9. Interview Question: How Did You Improve Speech-to-Text Accuracy?

Speech accuracy was improved by training Azure Speech with over 50 hours of real offshore audio. More than 200 safety-specific terms were added, along with noise cancellation techniques. Real-time feedback helped users record better audio, and a review screen allowed corrections. This significantly improved transcription accuracy and reliability.

10. Interview Question: How Did You Handle Cultural Resistance?

To address worker concerns, a “Just Culture” approach was introduced, focusing on learning rather than blame. Anonymous reporting was enabled, and investigation results were shared transparently. Workers were also recognized for reporting incidents, which reframed reporting as a positive behavior. This increased trust and significantly improved reporting adoption.

Final Thoughts

This project demonstrates how Power Platform and AI can transform safety operations from reactive reporting to proactive risk prevention. If you can clearly explain projects like this in interviews, you position yourself for senior roles. To strengthen your preparation, explore Power Fx interview questions and Canvas App fundamentals

For a deeper understanding of AI capabilities, refer to Microsoft AI Services documentation.

You can also explore how Power Platform integrates with AI in Microsoft Power Platform.