51 to 75 of 129 Permanent PySpark Jobs

Essential Skills Include: Proven leadership and mentoring experience in senior data engineering roles Expertise in Azure Data Factory, Azure Databricks, and lakehouse architecture Strong programming skills (Python, T-SQL, PySpark) and test-driven development Deep understanding of data security, compliance, and tools like Microsoft Purview Excellent communication and stakeholder management skills Experience with containerisation and orchestration (e.g., Kubernetes, Azure More ❯

We're looking for someone with strong technical expertise and a passion for solving complex business problems. You'll bring: Strong experience with SQL, SQL Server DB, Python, and PySpark Proficiency in Azure Data Factory, Databricks is a must, and Cloudsmith Background in data warehousing and data engineering Solid project management capabilities Outstanding communication skills, translating technical concepts into More ❯

MySQL, NoSQL) to AWS cloud. Strong knowledge of data extraction, transformation, and loading (ETL) processes, leveraging tools such as Talend, Informatica, Matillion, Pentaho, MuleSoft, Boomi, or scripting languages (Python, PySpark, SQL). Understanding of data warehousing and data modelling techniques (Star Schema, Snowflake Schema). Familiarity with security frameworks (GDPR, HIPAA, ISO 27001, NIST, SOX, PII) and AWS security More ❯

cloud-on prem hybrid solutions to optimize relevant workflow components for advanced artificial intelligence applications. The Contractor shall lead work to optimize cloud-based computing technologies, such as leveraging pyspark, distributed computation, and model training/inference, and integrate solutions into relevant delivery mechanisms or partner systems. The Contractor shall build tools and scripts to monitor, troubleshoot, automate, and More ❯

personalized digital interactions.Some Technologies We Work With:Python primarily, with bits and pieces of typescript and scalaGCP, AWS, Azure - in this order of relevanceGitHub, Docker, GitHub Actions, Terraform, KubernetesPandas, PySpark and SparkVertex AI, Azure OpenAI for LLMs Job Responsibilities Lead the execution of projects in a high-performing data science team, fostering professional growth and creating an inclusive and More ❯

cloud-on prem hybrid solutions to optimize relevant workflow components for advanced artificial intelligence applications. • The Candidate shall lead work to optimize cloud-based computing technologies, such as leveraging pyspark, distributed computation, and model training/inference, and integrate solutions into relevant delivery mechanisms or partner systems. • The Candidate shall build tools and scripts to monitor, troubleshoot, automate, and More ❯

Secret clearance is required for this position. Qualifications: To be considered for this role, you must have: Active Secret clearance or higher (required). Strong proficiency in Python, SQL, PySpark, and Java. At least 5 years of hands-on experience in pipeline engineering or data engineering roles. Demonstrated ability to perform root cause analysis and resolve production issues. Proven More ❯

managers to view all taskings, resolve duplication in requests, integrate with other tools, and provide metrics and reporting capabilities. The program is specifically seeking candidates with OpenTable format/PySpark experience, backend API technology such as Python (FastAPI), Java, SpringBoot, and RESTful design patterns best practices. Experience with JavaScript + UI framework (Angular, React, Vue, etc), Docker, AWS, and More ❯

Large and complex datasets Azure, Azure Databricks ML/Machine Learning/AI/Artificial Intelligence Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL Database Design Python/PySpark Azure Blob Storage Azure Data Factory More ❯

and influence decisions. Strong understanding of data models and analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear and adaptable communicator, able to More ❯

and stakeholder engagement abilities Strategic mindset with a focus on risk, governance, and transformation Proven ability to lead projects and coach others Technical skills to include: Python or R Pyspark Experience of deploying models AWS cloud Experience of GenAi Experience of working in a large complex organisation. Finance sector would be desirable, but not essential. Locations: London, Northampton, Manchester More ❯

Databricks, Synapse (Azure SQL DW), Cosmos DB, and Azure Data Lake. Knowledge of data testing concepts and best practices within data warehouse environments. Hands-on experience with Python/PySpark scripting for test development. Exposure to data testing automation is highly advantageous. Experience in the insurance domain is preferred. Excellent communication skills, both written and verbal. Previous experience in More ❯

Deliver end-to-end BI solutions using Azure Synapse, ADF, and Power BI (Datasets & Reports) Design and implement scalable data models and architectures Develop robust pipelines using SQL, Python (PySpark), and ideally other OOP languages Manage CI/CD processes using Git and modern deployment tools Debug, monitor, and optimize BI solutions across Microsoft's stack, including legacy tools More ❯

developing and deploying: data driven analytics; event driven analytics; set of analytics orchestrated through rules engine.• Experience documenting data models, schemas, data element dictionaries, and other technical specifications. • Pig• PySpark • Piranhas Desired:• Python• Java• Scala• Apache NiFi• Ansible• Experience working in Agile environment. • Experience deploying applications in a cloud environment• MongoDB Security Clearance Required: TS/SCI with Poly More ❯

term data strategy with a strong focus on data integrity and GDPR compliance To be successful in the role you will have Hands-on coding experience with Python or PySpark Proven expertise in building data pipelines using Azure Data Factory or Fabric Pipelines Solid experience with Azure technologies like Lakehouse Architecture, Data Lake, Delta Lake, and Azure Synapse Strong More ❯

term data strategy with a strong focus on data integrity and GDPR compliance To be successful in the role you will have Hands-on coding experience with Python or PySpark Proven expertise in building data pipelines using Azure Data Factory or Fabric Pipelines Solid experience with Azure technologies like Lakehouse Architecture, Data Lake, Delta Lake, and Azure Synapse Strong More ❯

like Data Factory, Databricks, Synapse, Cosmos DB, and Data Lake. Familiarity with data testing automation tools and frameworks. Domain knowledge in Insurance is a plus. Skilled in Python and PySpark for scripting and automation. Excellent written and verbal communication skills for effective collaboration. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their More ❯

Requirements: • 3+ years of hands-on experience as a Data Engineer working with Databricks and Apache Spark • Strong programming skills in Python, with experience in data manipulation libraries (e.g., PySpark, Spark SQL) • Experience with core components of the Databricks ecosystem: Databricks Workflows, Unity Catalog, and Delta Live Tables • Solid understanding of data warehousing principles, ETL/ELT processes, data … data engineering, and cloud technologies to continuously improve tools and approaches Technologies: AI AWS Azure CI/CD Cloud Databricks DevOps ETL GCP Support Machine Learning Power BI Python PySpark SQL Spark Terraform Unity GameDev Looker SAP More: NETCONOMY has grown over the past 20 years from a startup to a 500-people team working across 10 European locations More ❯

Experience in Cloud Data Pipelines Building cloud data pipelines involves using Azure native programming techniques such as PySpark or Scala and Databricks. These pipelines are essential for tasks like sourcing, enriching, and maintaining structured and unstructured data sets for analysis and reporting. They are also crucial for secondary tasks such as flow pipelines, streamlining AI model performance, and enhancing More ❯

You should be experienced in a data engineering role demonstrating a strong track record of designing, building, and maintaining data pipelines and data architectures. Required Skills - Proficiency in Python, Pyspark, SQL for data manipulation and querying. Experience with containerisation technologies, specifically Kubernetes and Docker. Proven experience in designing and implementing data pipelines, working with big data technologies and architectures. More ❯

Azure. Especially Synapse, ADF and Power BI (Datasets and Reports). Ideally SSIS, SSRS, SSAS with some understanding of Power App design and delivery Proficient in SQL and Python (PySpark) languages Understanding of data modelling concepts Experience of working with code management & deployment tools Proficient in debugging, monitoring, tuning and troubleshooting BI solutions. Knowledge and a proven track record More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯