51 to 70 of 70 Remote PySpark Jobs

domain, specializing in building scalable data models and delivering actionable insights for risk management, payments, and customer analytics. Skilled in Python (pandas, scikit-learn, PySpark), SQL, and data engineering pipelines to enable predictive and prescriptive analytics. Adept at working with large financial datasets, ensuring data quality, and collaborating with … business decisions. Demonstrated success in developing machine learning models for fraud detection, credit scoring, and customer segmentation. Key Skills: Languages/Tools: Python, SQL, PySpark, R, SAS ML/AI: Scikit-learn, TensorFlow, NLP, Time-Series Forecasting Data Engineering: ETL, Data Wrangling, Data Pipelines Cloud Platforms: AWS, Azure ...

data-driven decisions, while learning from experienced team members and developing your technical and business skills. Key Responsibilities: Build and maintain data pipelines, leveraging PySpark and/or Typescript within Foundry, to transform raw data into reliable, usable datasets. Familiarity with Palantir Foundry, PySpark, Kafka, TypeScript, PowerBI preferable. … years of experience building data pipelines at work or through internships is helpful. You can write clear and reliable Python/PySpark code. You are familiar with popular analytics tools (like pandas, numpy, matplotlib), big data frameworks (like Spark), and cloud services (like Palantir, AWS, Azure, or Google Cloud ...

that form the backbone of our Lakehouse platform, enabling accurate analytics, reporting, and regulatory compliance. You'll work with cutting-edge technologies including Databricks , PySpark , and Azure Data Factory , applying best practices in data engineering and governance to support this critical programme. Lead Databricks Engineer: Key Responsibilities Build … maintain Databricks pipelines (batch and incremental) using PySpark and SQL. Orchestrate end-to-end workflows with Azure Data Factory . Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming). Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets. Apply robust monitoring, logging ...

star schema and other best practices to optimize data usability. Sets best practices and standards on how we use SQL & dbt/Python/Pyspark and/or other languages to solve business problems. Becomes an expert in the business subject area the team are working on, holding expertise … risks, issues or control weaknesses that arise in your day-to-day What we’re looking for Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processing Extensive experience in dbt and ideally Airflow (or other orchestration tools) Extensive experience in Entity ...

include knowledge and use of at least six of the following: Informatica PowerCenter, Oracle PL/SQL, Business Objects, Tableau, Teradata, COBOL, Python, Spark, Pyspark, SQL Server, MS Access or MySQL. The employer will allow employee to work remotely from anywhere within the fifty United States and outlying territories. ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong ...

Your new company This is a pivotal opportunity to join the Data and Innovation division of a large complex organisation leading the delivery of SAM (Supervisory Analytics and Metrics)-a transformative programme enhancing supervisory decision ...

doing: Building and maintaining a unified data platform for global data processing. Developing reusable and scalable data solutions using Databricks, Python/PySpark, and Azure. Working closely with the data visualisation team to align back-end and front-end needs. Designing secure data access models and integrating external data … Strong understanding of data engineering principles, Lakehouse architecture, ETL/ELT, warehousing, and stream processing. Hands-on experience with Azure Databricks, Python/PySpark, SQL Server, and Azure Blob Storage. A track record of building secure, scalable data pipelines and solving complex problems. Great communication and documentation skills with ...

optimize Databricks Workspace, Clusters, Jobs, Repos, and Delta Live Tables. Develop modular ETL workflows using dbt and orchestrate them with Apache Airflow. Write advanced PySpark code, including UDFs and Pandas UDFs, for efficient data processing. Ensure data quality, reliability, and performance across all systems. Collaborate with data scientists, analysts … Support senior team members in identifying and addressing data science opportunities. Required Skills & Experience Proven experience with Databricks and its ecosystem Strong proficiency in PySpark, especially with UDFs and Pandas UDFs Hands-on experience with dbt and Airflow Databricks Certified Data Engineer Associate certification or willingness to achieve ...

optimize Databricks Workspace, Clusters, Jobs, Repos, and Delta Live Tables. Develop modular ETL workflows using dbt and orchestrate them with Apache Airflow. Write advanced PySpark code, including UDFs and Pandas UDFs, for efficient data processing. Ensure data quality, reliability, and performance across all systems. Collaborate with data scientists, analysts … Support senior team members in identifying and addressing data science opportunities. Required Skills & Experience Proven experience with Databricks and its ecosystem Strong proficiency in PySpark, especially with UDFs and Pandas UDFs Hands-on experience with dbt and Airflow Databricks Certified Data Engineer Associate certification or willingness to achieve ...

Design end-to-end data architectures using Medallion (Bronze/Silver/Gold) patterns Build metadata-driven ingestion pipelines and transformation frameworks Develop advanced PySpark/Spark SQL notebooks for data cleansing and modelling Create production-ready semantic models and support BI teams Implement governance, security, and CI/… clients to translate business requirements into technical solutions Essential Skills Strong experience with Microsoft Fabric workloads (Lakehouse, Data Factory, Pipelines, Notebooks, Semantic Models) Advanced PySpark/Spark SQL for large-scale transformations Integration with Dynamics 365, Dataverse, and/or Business Central CI/CD implementation using Azure DevOps ...

Data Engineer with ITIL New York City, NY 12 months Need local resumes with very strong Banking domain exp. Mandatory skills: Snowflake, SnowSQL, Python, Pyspark, ServiceNow, Jenkins, strong Banking Domain, ITIL Certification Job Description: Overview - We are seeking a skilled Data Engineer to join our dynamic team. The ideal … data engineering or related roles. Strong proficiency in writing complex queries for data extraction, transformation, and loading (ETL Experience in Python coding and PySpark frameworks. Strong experience in SQL Hands-on experience with designing, implementing, and managing datawarehouses. Deep understanding of Snowflake platform features, architecture, and best practices. Experience ...

Doing Designing end-to-end Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. Leading client workshops and guiding technical … technologies. What We're Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. CI/CD expertise with Azure DevOps ...

Data Engineer - Active SC, Databricks, PySpark Up to £475 per day (Inside IR35) Remote/London 6 months My client is an International Consultancy who are recruiting for an Data Engineer with Active Security Clearance (SC) and strong Databricks and Azure experience to deliver and optimise Data engineering solutions. … requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong experience with Databricks, PySpark and Delta Lake Expertise in Jobs & Workflows, cluster tuning, and performance optimisation Solid understanding of Data governance (Unity Catalog, Lineage, Access Policies) Hands-on with Azure services: Data Lake Storage (Gen2 ...

Data Engineer - Active SC, Databricks, PySpark Up to £475 per day (Inside IR35) Remote/London 6 months My client is an International Consultancy who are recruiting for an Data Engineer with Active Security Clearance (SC) and strong Databricks and Azure experience to deliver and optimise Data engineering solutions. … requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong experience with Databricks, PySpark and Delta Lake Expertise in Jobs & Workflows, cluster tuning, and performance optimisation Solid understanding of Data governance (Unity Catalog, Lineage, Access Policies) Hands-on with Azure services: Data Lake Storage (Gen2 ...

Pyspark Engineer- (Data Engineering, AWS GLUE) SC Cleared OR Eligible Stevenage (Hybrid) 2-3 days onsite Up to £80,000 High-impact programme - Revolutionary platform I am looking for a Security-Cleared Pyspark expert to take the reins on a range of highly ambitious Data Migration projects supporting … databases and AWS cloud environments. Strong hands-on experience with ETL processes and tools (Talend, Informatica, Matillion, Pentaho, MuleSoft, Boomi) or scripting using Python, PySpark, and SQL. Solid understanding of data warehousing and modelling techniques (Star Schema, Snowflake Schema). Familiarity with security frameworks such as GDPR, HIPAA ...

efficiency of our existing ML systems and processes. Build, administer and scale ML processing pipelines. Be comfortable navigating the following technology stack: Python3, Pyspark, scripting (Bash/Python), Hadoop, SQL, S3 etc. Should be able to understand internals of ML models such as Random Forest, CNN, Regression models … systems/engineering/related field 8+ years of experience in software engineering with a minimum of 4+ years in ML Good experience in Pyspark Expert level understanding of Python with design patterns and object-oriented programming. Experience debugging and reasoning about production issues is desirable. A good understanding ...

responsible for: Designing end-to-end data architecture aligned with modern best practices. Building and managing ingestion pipelines using Databricks and related tools. Developing PySpark/Spark SQL notebooks for complex transformations and cleansing. Applying governance, security, and CI/CD best practices across cloud environments. Leading technical discussions … producing professional documentation. To be successful in this role, you will have: Hands-on experience with Databricks including Unity Catalog. Strong PySpark/Spark SQL skills for large-scale transformations. Experience integrating with diverse data sources such as APIs, cloud storage and databases. Experience with the Azure cloud data ...

responsible for: Designing end-to-end data architecture aligned with modern best practices. Building and managing ingestion pipelines using Databricks and related tools. Developing PySpark/Spark SQL notebooks for complex transformations and cleansing. Applying governance, security, and CI/CD best practices across cloud environments. Leading technical discussions … producing professional documentation. To be successful in this role, you will have: Hands-on experience with Databricks including Unity Catalog. Strong PySpark/Spark SQL skills for large-scale transformations. Experience integrating with diverse data sources such as APIs, cloud storage and databases. Experience with the Azure cloud data ...