51 to 75 of 86 Remote PySpark Jobs

Spark, Azure Data Factory, Synapse Analytics). Proven experience in leading and managing a team of data engineers. Proficiency in programming languages such as PySpark, Python (with Pandas if no PySpark), T-SQL, and SparkSQL. Strong understanding of data modeling, ETL processes, and data warehousing concepts. Knowledge of More ❯

scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA … Understanding of how to build and run containerized applications (Docker, Helm) Familiarity with, or a working under-standing of big data search tools (Airflow, Pyspark, Trino, OpenSearch, Elastic, etc.) Desired Skills (Optional) Docker Jenkins Hadoop/Spark Kibana Kafka NiFi ElasticSearch About The DarkStar Group Our Company The DarkStar More ❯

delivering AI/ML projects end-to-end, with real business impact Hands-on Python skills including libraries like Scikit-learn, TensorFlow, Keras, and PySpark Strong understanding of model deployment and MLOps best practices Ability to work with real-world, unbalanced and messy datasets Skilled in translating technical work More ❯

delivering AI/ML projects end-to-end, with real business impact Hands-on Python skills including libraries like Scikit-learn, TensorFlow, Keras, and PySpark Strong understanding of model deployment and MLOps best practices Ability to work with real-world, unbalanced and messy datasets Skilled in translating technical work More ❯

Duration: Long Term Experience level: 10+ years Mandatory skills: ADF, Azure Data Bricks, PySpark, Spark SQL, PL/SQL, Python Job Description: Core Skills required ADF, Azure Data Bricks, PySpark, Spark SQL, PL/SQL, Python At least 5 years of consulting or client service delivery experience on … virtual machines, Container, Storage, ELB, AutoScaling is a plus Experience developing and deploying ETL solutions on Azure Strong in Power BI, Java, C , Spark, PySpark, Unix shell/Perl scripting Familiarity with the technology stack available in the industry for metadata management: Data Governance, Data Quality, MDM, Lineage, Data More ❯

scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA … each project including ETL mappings, code use guide, code location and access instructions. Required Skills Extract, Transform and Load (ETL) tools and processes Python, Pyspark, Pytorch AWS SQL APIs Linux Geospatial tools/data Desired Skills (Optional) Knowledge of agile methodologies/experience delivering on agile teams (Participates in More ❯

scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA … each project including ETL mappings, code use guide, code location and access instructions. Required Skills Extract, Transform and Load (ETL) tools and processes Python, Pyspark, Pytorch AWS SQL APIs Linux Geospatial tools/data Desired Skills (Optional) Knowledge of agile methodologies/experience delivering on agile teams (Participates in More ❯

Are you passionate about revolutionising engineering with AI? Here at Monolith AI we're on a mission to empower engineers to use AI to solve even their most intractable physics problems. We've doubled in size over the last four More ❯

Responsibilities: Analyse source data from source databases (PostgreSQL) to understand structures, relationships and semantics. Design and implement streaming data pipelines using AWS EMR and PySpark to generate real-time (fast-moving) features for the feature store. Develop and maintain batch processing pipelines using DBT and BigQuery to generate batch … ensure efficient integration into Feast feature store. Requirements Good knowledge of programming languages such as Python or Java. Strong experience with streaming technologies (Spark, PySpark, Flink, KSQL or similar) for developing data transformation pipelines. Solid understanding and practical experience with SQL and relational databases (PostgreSQL preferred). Proficiency with More ❯

Data Engineer Manager Department: Tech Hub Employment Type: Permanent Location: London Reporting To: Lead Data Engineer Description Contract type: Permanent Hours: Full time, 37.5 hours per week Salary: circa £80,000 depending on experience Location: Canary Wharf WFH policy: Employees More ❯

and supports the development of data pipelines to support model development. Proficient with software tools that develop data pipelines in a distributed computing environment (PySpark, GlueETL). Supports integration of model pipelines in a production environment. Develops understanding of SDLC for model production. Reviews pipeline designs, makes data model More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £85,000 - £100,000 - depending on experience Department: Technology and Data We're First Central Insurance & Technology Group (First Central for short), an innovative, market-leading insurance company. We protect More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £85,000 - £100,000 - depending on experience Department: Technology and Data We're First Central Insurance & Technology Group (First Central for short), an innovative, market-leading insurance company. We protect More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £85,000 - £100,000 - depending on experience Department: Technology and Data We're First Central Insurance & Technology Group (First Central for short), an innovative, market-leading insurance company. We protect More ❯

schemas (both JSON and Spark), schema management etc - Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python/PySpark frameworks - Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data. … Languages/Frameworks - JSON - YAML - Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.) - SQL - PySpark - Delta Lake - Bash (both CLI usage and scripting) - Git - Markdown - Scala (bonus, not compulsory) - Azure SQL Server as a HIVE Metastore (bonus) Technologies - Azure More ❯

WeDo has partnered with a leading fintech scale-up that is looking to scale it's Data Engineering Practice. Who are they.......Imagine a finance app that’s like your smart, low-fee wingman—budgeting help, savings boosts, and no confusing More ❯

Rate: Up to £600 per day 📆 Contract: 6 months (Outside IR35, potential to go perm) 🛠 Tech Stack: Azure Data Factory, Synapse, Databricks, Delta Lake, PySpark, Python, SQL, Event Hub, Azure ML, MLflow We’ve partnered with a new AI-first professional services consultancy that’s taking on the Big … and supporting team capability development What You Need: ✔ 5+ years in data engineering or backend cloud development ✔ Strong Python, SQL, and Databricks skills (especially PySpark & Delta Lake) ✔ Deep experience with Azure: Data Factory, Synapse, Event Hub, Azure Functions ✔ Understanding of MLOps tooling like MLflow and integration with AI pipelines More ❯

Rate: Up to £600 per day 📆 Contract: 6 months (Outside IR35, potential to go perm) 🛠 Tech Stack: Azure Data Factory, Synapse, Databricks, Delta Lake, PySpark, Python, SQL, Event Hub, Azure ML, MLflow We’ve partnered with a new AI-first professional services consultancy that’s taking on the Big … and supporting team capability development What You Need: ✔ 5+ years in data engineering or backend cloud development ✔ Strong Python, SQL, and Databricks skills (especially PySpark & Delta Lake) ✔ Deep experience with Azure: Data Factory, Synapse, Event Hub, Azure Functions ✔ Understanding of MLOps tooling like MLflow and integration with AI pipelines More ❯

complex data needs Developing and optimising data pipelines with Azure Data Factory and Azure Synapse Analytics Working with Spark notebooks in Microsoft Fabric, using PySpark, Spark SQL, and potentially some Scala Creating effective data models, reports, and dashboards in Power BI using DAX (and possibly M) Supporting data governance … structure is 90% SQL-based Basic familiarity with Python (we're all at beginner level, but it’s occasionally required) Openness to working with PySpark and potentially Scala (no prior Scala experience required, but a willingness to learn is appreciated) Comfortable working directly with clients and managing technical discussions More ❯

Senior Data Analyst - Pricing Data Engineering & Automation, CUO Global Pricing Let's care for tomorrow. Whether it's aircraft, international business, offshore wind parks or Hollywood film productions, Allianz Commercial has an extensive range of risks covered when it comes More ❯

Data Engineer. SC Clearance. Azure, Synapse, Pyspark, Power BI, SQL, Terraform up to £58k. Remote/London working pattern You will be an experienced Data Engineer who has worked within a SC Cleared environment within the last 12 months. This role is working with a rapidly growing consultancy who … Public Sector. You will have skills as follows: - Ideally 5 years+ Data Engineering experience - Azure Data Factory and Synapse data solution provision - Azure DevOps - Pyspark - Power BI - Python - Proficient in SQL - Good DATA Visualisation skills - Current SC ACTIVE VETTING - Working pattern of 4 days each week working remotely and … Rewards Portal and Company pension. Please forward a copy of your CV in the first instance. Summary: Data Engineer. Active SC Clearance. Azure, Synapse, Pyspark, Power BI, SQL, Terraform up to £58k. Remote/London working pattern More ❯

modern tech in a collaborative environment. What You’ll Do: Design, build, and maintain scalable data pipelines Optimize and automate data workflows Work with PySpark, Python, and SQL to process and manage large datasets Collaborate in a cloud-based environment to deliver efficient and reliable data solutions What We … re Looking For: Proven experience with Python, PySpark, and SQL Strong understanding of data engineering principles and cloud infrastructure Ability to work collaboratively and communicate technical concepts clearly A passion for clean, efficient, and scalable code Why Join Us? Supportive team environment with a strong focus on innovation Opportunities More ❯

e.g. Data Science, Mathematics, Statistics, Physics, Computer Science, Informatics or Engineering You have strong experience with analytics and data manipulation software e.g. R, Python, PySpark, SAS, SQL, SPSS You take a consultative approach and have polished communication and stakeholder management skills You're able to work independently and take … and inclusive environment Health and wellbeing support Volunteering opportunities Pension Apply now to find out more about this Data Scientist/Consultant (R Python PySpark SAS SQL SPSS) opportunity. At Client Server we believe in a diverse workplace that allows people to play to their strengths and continually learn. More ❯

e.g. Data Science, Mathematics, Statistics, Physics, Computer Science, Informatics or Engineering You have strong experience with analytics and data manipulation software e.g. R, Python, PySpark, SAS, SQL, SPSS You take a consultative approach and have polished communication and stakeholder management skills You're able to work independently and take … and inclusive environment Health and wellbeing support Volunteering opportunities Pension Apply now to find out more about this Data Scientist/Consultant (R Python PySpark SAS SQL SPSS) opportunity. At Client Server we believe in a diverse workplace that allows people to play to their strengths and continually learn. More ❯

e.g. Data Science, Mathematics, Statistics, Physics, Computer Science, Informatics or Engineering You have strong experience with analytics and data manipulation software e.g. R, Python, PySpark, SAS, SQL, SPSS You take a consultative approach and have polished communication and stakeholder management skills You're able to work independently and take … and inclusive environment Health and wellbeing support Volunteering opportunities Pension Apply now to find out more about this Data Scientist/Consultant (R Python PySpark SAS SQL SPSS) opportunity. At Client Server we believe in a diverse workplace that allows people to play to their strengths and continually learn. More ❯