76 to 100 of 161 PySpark Jobs

Databricks, Synapse (Azure SQL DW), Cosmos DB, and Azure Data Lake. Knowledge of data testing concepts and best practices within data warehouse environments. Hands-on experience with Python/PySpark scripting for test development. Exposure to data testing automation is highly advantageous. Experience in the insurance domain is preferred. Excellent communication skills, both written and verbal. Previous experience in More ❯

Deliver end-to-end BI solutions using Azure Synapse, ADF, and Power BI (Datasets & Reports) Design and implement scalable data models and architectures Develop robust pipelines using SQL, Python (PySpark), and ideally other OOP languages Manage CI/CD processes using Git and modern deployment tools Debug, monitor, and optimize BI solutions across Microsoft's stack, including legacy tools More ❯

developing and deploying: data driven analytics; event driven analytics; set of analytics orchestrated through rules engine.• Experience documenting data models, schemas, data element dictionaries, and other technical specifications. • Pig• PySpark • Piranhas Desired:• Python• Java• Scala• Apache NiFi• Ansible• Experience working in Agile environment. • Experience deploying applications in a cloud environment• MongoDB Security Clearance Required: TS/SCI with Poly More ❯

term data strategy with a strong focus on data integrity and GDPR compliance To be successful in the role you will have Hands-on coding experience with Python or PySpark Proven expertise in building data pipelines using Azure Data Factory or Fabric Pipelines Solid experience with Azure technologies like Lakehouse Architecture, Data Lake, Delta Lake, and Azure Synapse Strong More ❯

term data strategy with a strong focus on data integrity and GDPR compliance To be successful in the role you will have Hands-on coding experience with Python or PySpark Proven expertise in building data pipelines using Azure Data Factory or Fabric Pipelines Solid experience with Azure technologies like Lakehouse Architecture, Data Lake, Delta Lake, and Azure Synapse Strong More ❯

like Data Factory, Databricks, Synapse, Cosmos DB, and Data Lake. Familiarity with data testing automation tools and frameworks. Domain knowledge in Insurance is a plus. Skilled in Python and PySpark for scripting and automation. Excellent written and verbal communication skills for effective collaboration. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their More ❯

Have: Experience or Exposure in any one of these technologies. Power BI, Confluent Cloud Nice to Have: Exposure or experience in either of these technologies is beneficial. Python/PySpark/FiveTran Nice to Have: Exposure or experience is a plus. Strong SQL skills, including complex query writing (MS SQL Server) Good experience in data modeling In-depth knowledge More ❯

and manage environment promotion Maintain configuration, documentation, and monitoring Mentor and support Data Engineers Required Skills: Strong experience with Databricks , Azure Synapse , and Azure DevOps Proficient in SQL and PySpark Proven leadership in small engineering teams Skilled in configuration management and documentation Location: Warwickshire Type: Contract - 6 Months initially (high chance of extension) IR35 Status: Inside IR35 Rate: Open More ❯

and manage environment promotion Maintain configuration, documentation, and monitoring Mentor and support Data Engineers Required Skills: Strong experience with Databricks , Azure Synapse , and Azure DevOps Proficient in SQL and PySpark Proven leadership in small engineering teams Skilled in configuration management and documentation Location: Warwickshire Type: Contract - 6 Months initially (high chance of extension) IR35 Status: Inside IR35 Rate: Open More ❯

of Data Engineers. Essential Skills Extensive hands-on experience with Databricks - this is the core of the role. Strong background in Synapse and Azure DevOps. Proficiency in SQL and PySpark within a Databricks environment. Proven experience leading small engineering teams. Skilled in configuration management and technical documentation. If you're a Databricks expert looking for a role that blends More ❯

Requirements: • 3+ years of hands-on experience as a Data Engineer working with Databricks and Apache Spark • Strong programming skills in Python, with experience in data manipulation libraries (e.g., PySpark, Spark SQL) • Experience with core components of the Databricks ecosystem: Databricks Workflows, Unity Catalog, and Delta Live Tables • Solid understanding of data warehousing principles, ETL/ELT processes, data … data engineering, and cloud technologies to continuously improve tools and approaches Technologies: AI AWS Azure CI/CD Cloud Databricks DevOps ETL GCP Support Machine Learning Power BI Python PySpark SQL Spark Terraform Unity GameDev Looker SAP More: NETCONOMY has grown over the past 20 years from a startup to a 500-people team working across 10 European locations More ❯

setting up CI/CD pipelines (Azure DevOps, GitHub Actions, or GitLab CI targeting Azure) Good understanding of data in software applications, including experience with: Data libraries (Pandas, NumPy, PySpark) Data-driven solutions with MongoDB Building features that rely on performance monitoring, analytics, or metrics Start Date: ASAP Length: Initial 6 months Location: UK Remote IR 35 Status: Outside More ❯

Experience in Cloud Data Pipelines Building cloud data pipelines involves using Azure native programming techniques such as PySpark or Scala and Databricks. These pipelines are essential for tasks like sourcing, enriching, and maintaining structured and unstructured data sets for analysis and reporting. They are also crucial for secondary tasks such as flow pipelines, streamlining AI model performance, and enhancing More ❯

At least 2 years of hands-on experience developing Scala/spark pipelines At least 2 years of hands-on experience in handling large amount of big data using Pyspark Ecosystems. At least 1 years of experience with Python/Java required All applicants authorized to work in the United States are encouraged to apply Preferred Qualifications: Ability to More ❯

with machine learning frameworks, such as TensorFlow, PyTorch, or scikit-learn. Intermediate project management experience, with the ability to prioritize and manage competing demands. Advanced proficiency in Python, SQL, PySpark, or similar analytic tools. Strong problem-solving and analytical skills, with the ability to think strategically and balance technical depth with business acumen. Strong communication skills, both written and More ❯

engineering, math) related field or a similar quantitative analytics field. Up to 4 years of professional experience including exposure to data science concepts and computational tools (i.e., Python, Spark (PySpark), SQL, Hadoop, etc.). Professional, internship or project-based experience preferred. Familiarity with varying database structures and experience working with large datasets. Exposure to data visualization technology and capabilities More ❯

exchange platforms. Knowledge of dynamic pricing models. Experience with Databricks and using it for scalable data processing and machine learning workflows. Experience working with big data technologies (e.g., Spark, PySpark). Experience with online market research methods/products. Additional Information Our Values Collaboration is our superpower We uncover rich perspectives across the world Success happens together We deliver More ❯

You should be experienced in a data engineering role demonstrating a strong track record of designing, building, and maintaining data pipelines and data architectures. Required Skills - Proficiency in Python, Pyspark, SQL for data manipulation and querying. Experience with containerisation technologies, specifically Kubernetes and Docker. Proven experience in designing and implementing data pipelines, working with big data technologies and architectures. More ❯

AWS/Azure - moving towards Azure). Collaborate with stakeholders and technical teams to deliver solutions that support business growth. Skills & Experience Required: Strong hands-on experience in Python, PySpark, SQL, Jupiter . Experience in Machine Learning engineering or data-focused development. Exposure to working in cloud platforms (AWS/Azure) . Ability to collaborate effectively with senior engineers More ❯

Azure. Especially Synapse, ADF and Power BI (Datasets and Reports). Ideally SSIS, SSRS, SSAS with some understanding of Power App design and delivery Proficient in SQL and Python (PySpark) languages Understanding of data modelling concepts Experience of working with code management & deployment tools Proficient in debugging, monitoring, tuning and troubleshooting BI solutions. Knowledge and a proven track record More ❯

Senior Engineering Level Mentoring/Team Leading experience - Nice to have (Full Support from Engineering Manager) Hands on development/engineering background Machine Learning or Data background Technical Experience: PySpark, Python, SQL, Jupiter Cloud: AWS, Azure (Cloud Environment) - Moving towards Azure Nice to Have: Astro/Airflow, Notebook Reasonable Adjustments: Respect and equality are core values to us. We More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯

on experience or strong interest in working with Foundry as a core platform Forward-Deployed Engineering – delivering real time solutions alongside users and stakeholders Broader Skillsets of Interest: Python & PySpark – for data engineering and workflow automation Platform Engineering – building and maintaining scalable, resilient infrastructure Cloud (AWS preferred) – deploying and managing services in secure environments Security Engineering & Access Control – designing More ❯