Reimagining Data: The AI-Driven Transformation

Last week, our bem team hosted a panel with industry experts Karan Goel, CEO of Cartesia, Jordan Tigani, CEO of Motherduck, and Tomasz Tunguz, GP at Theory for an evening of discussions around the future of data, AI, and infrastructure within mid-sized and enterprise organizations. 

Since space was limited, we’re highlighting below our four key thematic takeaways from the discussion, and if you’d like to join us for the next event, follow along on Linkedin so you can stay up to date. 

Bottom Line: Data infrastructure is no longer just a cost center—it's a strategic differentiator. The data infrastructure inflection point is upon us

Technology leaders from startups, mid-sized orgs, and enterprises are facing a critical moment of reckoning: AI is fundamentally challenging traditional data architectures, exposing long-standing infrastructural limitations and creating unprecedented opportunities for transformation.

The economic imperative

The numbers tell a compelling story:

• 50% of enterprise AI deployments now use small language models

• 75-90% reduction in latency

• Up to 600x lower inference costs

1. The semantic layer: Your critical missing link

The most profound insight from industry leaders, both from the panel and attendees, was the glaring gap in data infrastructure. Jordan Tigani from MotherDuck highlighted that AI has dramatically stressed the lack of a semantic layer in organizational data ecosystems. This isn't just a technical nuance—it's a strategic vulnerability for any organization.

Why is this a challenge for many companies?

• Semantic layers have historically been difficult to sell as standalone products

• Combining reasoning capabilities with state management remains a fundamental challenge

• Existing data storage approaches are struggling to keep pace with AI demands

2. AI's impact on data architecture

Beyond traditional infrastructure

Our panelists uncovered another stark assessment: AI hasn't significantly changed data storage approaches—yet. This presents a massive opportunity for forward-thinking organizations.

Strategic focus areas we expect to see more of in 2025:

• Companies developing flexible data modeling capabilities

• Creating adaptable semantic layers

• Enabling intelligent data routing and reasoning

Emerging data deployment models

The landscape is shifting dramatically:

• On-premise deployments are re-surging

• Local hardware capabilities are dramatically underutilized

• Enterprises are moving away from pure SaaS-dominated paradigms

3. Data security: A new paradigm

AI is rewriting the rules of data security and access:

• Three critical security categories have emerged:

  1. Guardrails

  2. Data loss prevention

  3. Model poisoning prevention

• Sophisticated organizations are treating AI agents like human employees

• Granular identity and access management is becoming crucial

4. Practical implementation strategy

Building your AI-ready data infrastructure

1. Semantic layer development

   • Invest in tools like Malloy for semantic modeling

   • Create flexible data interpretation frameworks

   • Build adaptable reasoning capabilities

2. Deployment optimization

   • Security and governance

     • Develop role-based access controls

     • Create AI-specific security protocols

     • Implement comprehensive data loss prevention

   • Explore "constellations of models" approach

   • Leverage small language models for efficiency

   • Implement hybrid deployment strategies

Looking Forward

The most successful organizations will be those that treat data as a dynamic, intelligent asset, build flexible, adaptive infrastructures, and prioritize semantic understanding over raw data storage.

The future belongs to those who can transform data from a static resource to an intelligent, responsive ecosystem. Chat with our bem team to learn more about how we approach this within data transformation, contextual analysis, and routing.