In the Architecture, Engineering, and Construction (AEC) industry, effective use of agentic AI systems can transform workflows, enhance knowledge accessibility, and automate complex tasks. Agents can act as intelligent intermediaries, dynamically directing processes and leveraging multimodal capabilities to boost productivity and inform decision-making.

For instance, these capabilities allow agents to cross-reference technical drawings with material specifications, identify discrepancies in designs, and streamline project documentation workflows. By integrating insights from diverse data formats, agents empower teams to make informed decisions quickly and accurately. The success of these agents hinges on designing systems that respect the unique requirements of the AEC industry—including data privacy, secure environments, and seamless integration with industry-standard authoring tools like Revit, Rhino, and AutoCAD.

Understanding Agentic Systems

Agentic systems can be categorized into workflows and agents:

• Workflows: These are predefined systems where tools and LLMs follow structured paths to complete tasks. For example, workflows in the AEC industry might automate the generation of compliance reports by sequentially extracting data from specifications and validating it against regulatory standards.

• Agents: These systems dynamically plan and execute tasks, adjusting to feedback from tools and the environment. An agent in the AEC context could analyze ambiguous design requirements, propose solutions, and iterate based on user feedback or project constraints.

Both approaches have their place depending on the specific use case. Workflows suit predictable, repetitive tasks such as extracting structured data from drawings for regulatory compliance reports. Agents, however, excel in complex, open-ended scenarios—like interpreting ambiguous design requirements or synthesizing insights across diverse data sources and knowledge bases.

When to Use Agents

Not every challenge in AEC requires agentic systems. Single LLM calls with retrieval or in-context examples may suffice for straightforward tasks such as generating a project brief summary or suggesting a selection of appropriate finishes for exterior facade cladding from a product database. Agentic systems become valuable when:

1. Flexibility is needed: Tasks like writing specification documents or RFP responses that require multiple iterations and refinement.

2. Tasks involve diverse data formats: Combining textual and visual data sources in complex knowledge search and retrieval scenarios.

3. Complex automation: Automating design iterations and software-specific UIs such as Revit and Rhino using specialized coding agents integrated with their SDKs and scripting languages.

   
   

Building Blocks of Agentic Systems

1. Augmented LLMs: LLMs enriched with tools, retrieval, and memory capabilities form the foundation. This might include:

   • Retrieval: Searching material specifications or past project archives.

   • Tools: Interfacing with third-party APIs to enhance analysis and reasoning. For example, retrieving supply chain and product data for lifecycle carbon analysis.

   • Memory: Retaining context across multi-step processes and/or conversations.

2. Prompt Chaining: Breaking tasks into sequences of LLM calls. For example:

   • Generating a project plan outline.

   • Validating it against compliance criteria.

   • Review draft.

   • Finalizing details.

3. Routing: Directing inputs to specialized tasks. In AEC, this could involve:

   • Classifying queries into structural, architectural, electrical, or mechanical categories for targeted retrieval.

4. Parallelization: Allowing LLMs to process multiple subtasks concurrently. For example, an agent or team of specialized agents simultaneously evaluates a design option against multiple criteria and reference data to optimize for cost, sustainability, and compliance.

5. Orchestration Agent: A central LLM delegates tasks to other specialized workers. For instance, orchestrating a clash detection process where one agent identifies conflicts in a 3D model while another proposes solutions.

6. Self-Reflection and Improvement: This might involve refining design outputs based on structural analysis feedback from a simulation, or improving the generation of a proposal draft.

Enhancing Searchability with Multimodal Capabilities

The AEC industry’s reliance on visual documents necessitates multimodal capabilities and systems. For example, technical drawings and 2D details can be processed to extract metadata, identify design elements, and link them to textual specifications, creating a comprehensive understanding of project requirements. This enables more effective cross-referencing and ensures accuracy in construction planning and execution. These include:

• Embedding Techniques: Utilizing models to create embeddings for technical drawings, enabling similarity searches.

• Multimodal Search and Retrieval: Linking textual specifications with corresponding CAD drawings.

• Vision-Language Models: Leveraging technologies like Vision Transformers to extract information from schematics and annotate drawings automatically.

Compliance and Regulatory Requirements

• Agents can interpret building codes and cross-reference them with project documentation to ensure compliance.

• Automating permit submission processes by extracting required data from plans and specifications.

Knowledge Search

• Creating a unified knowledge base allows agents to answer complex queries about past projects, including cost analysis, design decisions, and material performance.

• Multimodal agents can search across textual and visual documents to identify precedents and best practices.

Automation of Design Software

• Using agents integrated with Revit, Rhino, or AutoCAD SDKs to perform repetitive tasks like updating families, generating parametric designs, or conducting energy simulations.

• Coding agents can automate script generation for Dynamo in Revit or Grasshopper in Rhino, assisting technical users in creating and extending custom tools.

Secure Environments and Data Privacy

In the AEC industry, project and client information are highly sensitive, including details like project financials, proprietary design specifications, and contractual agreements. These data types require measures to ensure confidentiality and compliance with industry standards. When Implementing agentic systems within data-sensitive environments it's important to consider the following:

• Data Anonymization: Ensuring agents handle anonymized or pseudonymized data to protect client identities.

• Secure Architecture: Deploying agents within private cloud environments or on-premise setups.

• Controlled Tool Access: Restricting tools to predefined scopes to prevent unauthorized data access or modifications.

• Role-based Knolwedge Access: Controlling access

Key Design Principles for AEC Agents

1. Simplicity: Start with straightforward implementations, adding complexity only as needed.

2. Transparency: Design agents to expose their decision-making processes, enabling easier debugging and trust.

3. Thorough Documentation: Clear documentation for tools and APIs ensures agents understand their functionalities fully.

4. Iterative Testing: Deploy agents in sandboxed environments to evaluate performance, particularly for error recovery and data handling.

Use Cases

To demonstrate how agentic systems can be leveraged for domain-specific use cases, AEC Foundry has developed the following tools:

1. SWMS AI: Our AI tool for generating health and safety risk assessments employs an agentic RAG system in the backend, featuring multiple agents specialized in hazard identification and classification, control measure definition, risk scoring, and a self-reflection mechanism to refine assessments automatically. Learn more.

2. Archie: Our AI agent for building codes and standards employs a multimodal agentic system with access to building code knowledge bases. Using query routing and vision models, Archie performs deep searches across multiple documents to answer multi-hop queries. It also uses vision capabilities to enable users to analyze technical drawings in the context of regulatory compliance. Learn more.

3. Upsafe: Our safety observation platform leverages fine-tuned language models to classify and categorize safety observations and provide real-time translation to support diverse workforces. Learn more.

Conclusion

The integration of agentic AI systems in the AEC industry offers significant potential for transforming workflows, enhancing searchability, and automating complex processes. By leveraging multimodal capabilities and prioritizing secure, privacy-preserving environments, organizations can unlock efficiencies and gain a competitive edge. However, the key to success lies in tailoring these systems to the unique challenges and opportunities of the AEC domain.