251 to 275 of 395 PySpark Jobs

well as familiarity with software engineering best practices (such as version control, unit testing, CI/CD) and big data and cloud technologies (PySpark preferred). Experience training and deploying deep learning models in cloud environments (including GPU‐enabled workflows) is a strong advantage. What ...

with strong identity and networking controls and work with cross-functional teams to address complex requirements. You will demonstrate live coding in Python and PySpark, contribute to platform adoption, and influence product direction. #J-18808-Ljbffr ...

London seeks a Senior Data Engineer to design, build, and operate enterprise-scale data solutions. You will leverage Snowflake with hands-on Python, PySpark, AWS and Airflow to create scalable data platforms and robust analytics capabilities. The role emphasizes technical leadership, governance, and collaboration with cross-functional teams ...

collaborate with cross‐functional teams to deliver customer outcomes. Candidates typically have 4+ years in solutions architecture or related roles, strong Python and PySpark skills, and willingness to #J-18808-Ljbffr ...

through full lifecycle from design to production deployment, architecting secure, scalable solutions with a focus on IAM, networking, and platform administration. Strong Python/PySpark skills and a track record in data architectures are essential. #J-18808-Ljbffr ...

Dataiku platform every day. Aside from the visual tools, our team mostly uses Python and SQL, with occasional work in other languages (e.g., R, PySpark, JavaScript). An ideal candidate is excited to learn complex new technologies and modelling techniques and to explain their work to other data scientists ...

Experience working with stakeholders and translating business challenges into analytical solutions. Previous experience mentoring or supporting less experienced team members. Exposure to Databricks and PySpark would be beneficial. What They Offer Salary up to £52,000. Dedicated learning and development time each week. Training budget to support continuous growth. ...

unsupervised techniques Experience with tree-based and boosting algorithms such as XGBoost, LightGBM, and related approaches Experience working with large-scale datasets, ideally using PySpark Strong analytical and SQL skills for handling, querying, and interpreting complex datasets Ability to communicate insights effectively through visualisations using tools such as Matplotlib ...

unsupervised techniques Experience with tree-based and boosting algorithms such as XGBoost, LightGBM, and related approaches Experience working with large-scale datasets, ideally using PySpark Strong analytical and SQL skills for handling, querying, and interpreting complex datasets Ability to communicate insights effectively through visualisations using tools such as Matplotlib ...

closely with SMEs and the Data Architect to shape our data management strategy and meet business needs. Ideal candidates bring deep experience with Databricks, PySpark, Azure ecosystems, and ELT pipeline development, with an emphasis on data modelling and effective collaboration across IT and business teams. #J-18808-Ljbffr ...

performing engineering squad and own technical challenges from design through production, driving best practices and engineering standards. You'll work with Azure, Databricks and PySpark to deliver scalable ETL pipelines and data platforms that power critical business decisions. #J-18808-Ljbffr ...

work. The contract is initially 6 months with remote working. You will own ingestion, model deployment, evaluation, and CI/CD across environments, leveraging PySpark, Unity Catalog, and batch inference on Claude. #J-18808-Ljbffr ...

IR35 . Rate: £650 - £750 Location: Hybrid – 2/3 days per week on-site Contract: 6 months | Outside IR35 Key Skills: Data Lakehouse PySpark DBT Iceberg AWS Strong data engineering and pipeline development experience You’ll be responsible for designing, building and optimising scalable data pipelines and lakehouse ...

with the ability to think beyond routine approaches to solve technical problems. Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, and Databricks on AWS, processing and transforming large-scale financial datasets for analytics and reporting. Develop fact and dimension data models for reporting … qualifications, capabilities, and skills Proven experience in data management, ETL/ELT pipeline development, and large-scale data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with ...

with the ability to think beyond routine approaches to solve technical problems. Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, and Databricks on AWS, processing and transforming large-scale financial datasets for analytics and reporting. Develop fact and dimension data models for reporting … qualifications, capabilities, and skills Proven experience in data management, ETL/ELT pipeline development, and large-scale data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with ...

/ELT patterns, data modeling (dimensional + normalized), and batch scheduling dependencies. Ability to interpret data transformation logic from pipelines (Spark SQL/PySpark/Hive queries/orchestration configs). Strong documentation capability: source‐to‐target mappings, lineage diagrams, data dictionaries, metadata standards, and control evidence packs. Technical … Skills Strong proficiency in Python (data analysis/automation for metadata extraction, validation scripts, rule checks). Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

including architecture, development, deployment, and optimization. Define Databricks workspace architecture, clusterstrategies, security models, and operational standards. Design and developscalable data pipelinesusing Databricks, Apache Spark, PySpark, SQL, andDelta Lake . Implement data ingestionframeworks supporting batchand real-timedata processing. Design Delta Lake architectures including: Delta tables Medallion architecture (Bronze/Silver … reviews. Skills & Experience Strong experience in dataengineering, big data, or solution architecture. Strong hands-onexperience with Databricks Platform . Expert knowledge of: Apache Spark PySpark/Scala Delta Lake SQL Data Lakehouse architecture ETL/ELT development Experience designing anddelivering enterprise Databrickssolutions. Strong experience withcloud platforms: Azure Databricks (preferred ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

data modelling, semantic layers and Power BI delivery. You'll have the opportunity to: ·Design and build production-grade pipelines using Azure Databricks, Python, PySpark and SQL ·Develop scalable Lakehouse architecture using Delta Lake and Unity Catalog ·Build data and BI products from scratch and own them through … Claude and OpenRouter to support development, code review and documentation To succeed, you'll need: ·Strong production experience with Azure Databricks ·Advanced Python, PySpark and SQL skills ·Advanced Power BI, DAX and data-modelling experience ·Experience building and owning complex data or BI solutions end to end ·Confidence translating ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

Function Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - Edinburgh, London, Manchester Skills Python & SQL: Advanced Python (PySpark GlobalLogic is seeking a Data & AI Architect working in our UK & Ireland Go to Market Team to drive pipeline growth and lead enterprise transformation pursuits across … mindset, with experience working across geographies including India, East Europe and North America. Proven track record of acceleration of Business growth Deep expertise in PySpark/Python, Advanced SQL, data modeling, batch/real-time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge ...

Function Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - Edinburgh, London, Manchester Skills Python & SQL: Advanced Python (PySpark GlobalLogic is seeking a Data & AI Architect working in our UK & Ireland Go to Market Team to drive pipeline growth and lead enterprise transformation pursuits across … mindset, with experience working across geographies including India, East Europe and North America. Proven track record of acceleration of Business growth Deep expertise in PySpark/Python, Advanced SQL, data modeling, batch/real-time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge ...

consulting and technology services. The candidate will design, build, and maintain scalable data pipelines, data lakes, and data warehouse solutions on AWS, using PySpark, Spark, Python, and AWS services to enable #J-18808-Ljbffr ...