Large Data Volumes (LDV) & Indexing: Architecting SOQL Queries for 5M+ Records in Salesforce & OmniStudio Development

Large Data Volumes (LDV) & Indexing: Surviving the 5-Million Record Threshold

Querying 10,000 records is trivial. Querying 5,000,000 records is where systems break, architectures fail, and user experience crumbles.

When your enterprise data model crosses into the territory of Large Data Volumes (LDV), the rules of the game change entirely. A SOQL query that worked perfectly in Sandbox will suddenly trigger the dreaded System.QueryException: Non-selective query against large object type in Production. In the UI, users stare at infinite loading spinners until the transaction violently times out.

Whether you are pulling data into a Lightning Web Component or running a heavy DataRaptor Extract in OmniStudio, the platform will relentlessly enforce its Governor Limits. To architect solutions that scale seamlessly, you must shift your mindset from "querying data" to "designing selectivity."

Here is the architectural blueprint for conquering LDV and ensuring your queries and reports never fail.

1. The Selectivity Threshold: Why Your Queries Fail

Salesforce does not let you perform full-table scans on massive objects. If a SOQL query takes too long, it consumes shared CPU resources, which is strictly prohibited in a multi-tenant environment.

To prevent this, Salesforce requires queries against large objects (typically > 100,000 records) to be selective. A query is selective when one of the query filters targets an indexed field and the filter returns a small enough portion of the total data.

The Golden Rules of Selectivity:

• Standard Indexes (Id, Name, Lookup, Email): The filter must return less than 30% of the first million records, and 15% of records after the first million.

• Custom Indexes (External ID, Unique, or Custom requested via Support): The filter must return less than 10% of the first million records, and 5% of records after the first million (up to a maximum of 333,333 records).

If your WHERE clause relies on unindexed fields, or if your filter returns too many rows, the platform's Query Optimizer abandons the index, attempts a full table scan, and ultimately throws a timeout error.

2. Architectural Strategies for LDV

To navigate around these hard limits, architects must proactively design the data layer for high-velocity retrieval.

3. LDV in OmniStudio Architecture

In OmniStudio, non-selective queries are silent killers. When a FlexCard calls an Integration Procedure that triggers a DataRaptor Extract against a 5-million-row Order object, an unindexed WHERE clause will break the entire UI rendering process.

Architectural Fix: Never use wildcard LIKE '%value%' statements in DataRaptors dealing with LDV, as this forces a full table scan. Always ensure that the parameters passed from the OmniScript or FlexCard correspond directly to indexed fields (Standard, External IDs, or Custom Indexes). If necessary, aggregate data beforehand using Batch Apex or CRM Analytics to feed pre-calculated data into your OmniStudio components.

4. Front-End Execution: Searching LDV Safely with LWC

When building custom UI components to interact with massive tables, you must never attempt to load bulk lists. Instead, force the user to provide selective parameters, and utilize LWC with the @wire service to execute clean, indexed queries without lagging the browser.

Here is an optimized LWC controller fetching data safely from a massive table using indexed parameters:

Final Verdict

Designing for Large Data Volumes requires anticipating failure. Do not wait for your production org to hit 5 million records before thinking about selectivity. Architect your SOQL queries, OmniStudio DataRaptors, and custom indexes from day one to ensure that no matter how vast your catalog or transaction history becomes, your user experience remains lightning fast.

Connect

Reach out for feedback or technical inquiries.

Email

andre@theforcejournal.tech

© 2026. All rights reserved.