Client Background
A mid-sized, fast-growing pharma company engaged us to stand up a modern data platform. The team had 20+ data engineers and needed strong governance, easy cross-domain sharing, and a foundation that scales.
Aspiration
Stand up a new platform on Databricks and dbt with solid DevOps and access control, enable efficient sharing across domains, and support rapid onboarding and collaboration.
How We Did It
- Platform architecture on Databricks: Provisioned workspaces, catalogs, and permissions with Terraform for consistent, compliant environments.
- Transformation with dbt: Modular projects, tests, and documentation, with shared packages to reuse models across domains.
- DevOps and CI/CD: Branching strategy, environment promotion, automated testing, and quality gates baked into pipelines.
Rollout framework:
- Strategy and alignment workshops with business and tech
- Technical discovery with domain teams
- Foundation rollout for infra, security, CI/CD, and branching
- Enablement and onboarding for users and developers
- Continuous improvement for migration, maintenance, and support
Migration at scale: An agentic migration assistant to port MS SQL objects into dbt, first 1:1, then automated refactoring, delivering an estimated 80% speed-up versus prior manual approaches.
Key Outcomes
- Governed and scalable: A single, well-controlled platform that reduces data silos and simplifies access.
- Cross-domain sharing: Standardized models and packages make inter-domain collaboration straightforward.
- Faster delivery: Migration and development velocity improved significantly, with up to 80% faster migrations.
- Future-ready: A strong base for analytics, ML, and automation across multiple domains.
Want similar results?
Let's discuss how we can help transform your data capabilities.