Databricks Workspace Enablement Lead

Databricks Workspace Enablement Lead

Hartofrd CT Hybrid

Engagement type Statement of Work SOW short duration defined outcomes with potential to transition to ongoing execution support

CONTEXT CURRENT STATE

Bains Polaris 20 program is a firmwide data platform transformation built on Databricks with a medallion architecture Bronze Silver Gold and Unity Catalog governance The platform is in active execution multiple data domains are progressing through Silver ingestion with Gold layer certification underway for priority business products including the Commercial Dashboard Revenue Reporting and GROM

While the platform infrastructure is established and delivery is accelerating we do not yet have consistent repeatable operational processes in place to support the growing footprint The program needs an experienced Databricks practitioner to bring structure and standards to how the workspace is managed and monitored creating the operational backbone that the team and downstream business users will rely on

Currently we have one custom application in production pulling from Databricks with several data owners accessing the gold layer to certify data We have already seen early requests to get into the platform and the transparency and enablement doesnt seem to be sustainable Our business goal by end of the year is to have all functional teams on the workspace and support requests by report writers and data consumers to leverage our certified enterprise data We need a clean foundation that is scalable and low overhead to manage

SCOPE OF WORK

The Workspace Enablement Lead will design and implement the operational standards and tooling required to run Polaris 20 at scale Core deliverables include

Workspace governance setup Workspace governance setup

Define and enforce naming conventions folder structures and access control patterns within Unity Catalog

Set up and standardize cluster policies compute configurations and cost allocation tagging

Provide best practices around data request access processes data consumer data owner

Data quality standards Data quality standards

Implement automated quality checks integrated into the ingestion pipeline

Establish escalation and remediation workflows when quality checks fail

Monitoring observability dashboards Monitoring observability dashboards

Build operational dashboards covering pipeline health SLA performance compute costs and data freshness

Set up proactive ing for platform engineering and data ownership teams

Define KPIs and SLAs for platform operations eg ETL run time uptime targets

Handoff knowledge transfer Handoff knowledge transfer

Produce comprehensive documentation for all standards templates and configurations

Conduct structured handoff to the internal team andor incoming maintenance resource

IDEAL PROFILE

7 years handson Databricks platform oversight experience Unity Catalog Delta Lake DLT SQL Warehouses

Demonstrated experience building operational standards and governance frameworks not delivering pipelines

Experience with monitoring and observability in data platforms ing dashboards SLA management

Excellent documentation and communication skills this role produces artifacts others will use long after the engagement ends

Prior professional services or management consulting environment experience is a plus