226 to 250 of 372 Apache Spark Jobs in England

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

demonstrable experience of working in rapidly changing client environments. Needs to be a practitioner and proponent of Agile and Dev Ops Proficiency in Hadoop, Spark, Scala, Python, or a programming language associated with data engineering Expertise in building and deploying production level data solutions maintained by application support teams ...

audiences both written and verbal Preferred (but not required) to have: Hands on experience with Python Experience working with modern data technology (e.g. dbt, spark, containers, devops tooling, orchestration tools, git, etc.) Experience with data science and machine learning technology People want to buy from people who understand them. ...

. Experience with Natural Language Processing (NLP), Large Language Models (LLMs) or Image Processing. Experience in areas of model development, data processing and streaming (Spark, Kafka), microservices in python (Flask or FastAPI), and interactive visualisations and User Interfaces (Streamlit, Plotly, Gradio etc). While we think the above experience ...

machine learning experience (e.g. python, tensorflow, pytorch). Familiarity with SQL and large-scale data processing technologies such as Hadoop, Hive, Impala, Scala, Spark etc. Technology leadership experience from defining market driven visions/goals to the implementation of solutions in a production environment in collaboration with cross-functional ...

translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with an ethos of commitment ...

performance evaluation. Data & Solution Development Gather, preprocess, analyze, and model large volumes of structured and unstructured data from multiple sources. Use Python, SQL, Spark, and other relevant technologies to build scalable analytical and machine learning solutions. Develop features, datasets, and analytical approaches that support model development, experimentation, and evaluation. ...

delivered dataset, model or capability, including deciding what was good enough. Experience with a modern data stack comparable to ours: a cloud data lake, Spark or Databricks, and AWS. Clear communication with technical and non technical stakeholders, and the ability to cascade direction to a team so that each ...

Engineer, maintaining and further developing a well-established cloud and CI/CD setup to implement and evolve reliable, well-tested data pipelines in Spark/Scala and to extend the dimensional data models. You will be part of the technical team collaborating closely with platform engineering, data science … ingesting from various sources into Databricks Collaborate closely with software engineers to evolve our software platform and address technical challenges Design, develop, and optimize Spark/Scala jobs for ingestion and transformation, using the existing GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests ...

integration, and production support. Good understanding of ETL/ELT, data warehousing, data lakes, dimensional modelling, batch processing, and stream processing. Experience with Apache Beam, Kafka, REST APIs, Docker, Kubernetes, or GKE. Experience with data formats such as JSON, CSV, Avro, and Parquet. Working knowledge of Git, automated testing … observability, and release management. Ability to design solutions for scalability, resilience, security, performance, operability, and cost efficiency. Preferred Skills Experience with Cloud Composer or Apache Airflow, Dataproc or Spark, Dataform or dbt, and metadata-driven pipeline frameworks. Exposure to Terraform or another infrastructure-as-code tool. Knowledge ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with … platforms such as Amazon Bedrock. Experience developing data-based apps in Streamlit/Shiny Skills in ‘Big Data’ technology tools like Hadoop/Spark Knowledge of Infrastructure as Code tools like CloudFormation/Terraform If you can bring some of these skills and experiences, along with transferable strengths ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with … platforms such as Amazon Bedrock. Experience developing data-based apps in Streamlit/Shiny Skills in ‘Big Data’ technology tools like Hadoop/Spark Knowledge of Infrastructure as Code tools like CloudFormation/Terraform If you can bring some of these skills and experiences, along with transferable strengths ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with … platforms such as Amazon Bedrock. Experience developing data-based apps in Streamlit/Shiny Skills in ‘Big Data’ technology tools like Hadoop/Spark Knowledge of Infrastructure as Code tools like CloudFormation/Terraform If you can bring some of these skills and experiences, along with transferable strengths ...

Responsibilities Azure Platform Support & Monitoring Support and maintain Azure-based data solutions, including Azure Data Factory pipelines, datasets, linked services, triggers; Azure Databricks (Spark jobs, notebooks, clusters); Azure Machine Learning models and endpoints; Power BI dashboards and dataset refreshes. Monitor and troubleshoot failures in pipelines, jobs, and ML workflows … detailed SOPs, runbooks, knowledge‐base articles, and client documentation. Required Skills and Qualifications Azure Services – Azure Data Factory (pipelines, triggers, parameterization, monitoring); Azure Databricks (Spark, notebooks, job orchestration); Azure Machine Learning (pipelines, model deployment, monitoring); Power BI Service (dataset refreshes, access control, report diagnostics). DevOps & Automation – CI/ ...

components . Build and maintain platform capabilities supporting batch and real-time data workloads . Develop and operate distributed data services using Kafka, Kinesis, Spark, Flink or equivalent technologies . Build and maintain cloud-native data platform capabilities using AWS and/or Azure . Support data ingestion, processing … Hands-on experience with real-time streaming and event-driven architectures , including Kafka, Kinesis or equivalent. Experience with distributed data processing technologies such as Spark, Flink, Kafka Streams or equivalent. Strong experience with AWS and/or Azure cloud environments. Experience with cloud services such as S3, Glue, Lambda ...

Senior Data Engineer – SQL/Python/AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data ...

meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes Big Data Analytics platforms Cloud-based solutions experience AWS preferred Software Engineering Expertise with: RDBMS … Knowledge of: MLOps Feature Engineering Exposure to data analysis and ML tools such as: scikit-learn pandas NumPy XGBoost LightGBM Familiarity with orchestration tools: Apache Oozie Apache Airflow Why you’ll love it here We want to make Prima a happy and empowering place to work. ...

applying AWS principles for scalable solutions. Ideal candidates should have strong skills in Python, Scala, or Java, and experience with Big Data technologies like Spark and Hadoop. This opportunity includes comprehensive benefits and a commitment to inclusivity and professional growth. #J-18808-Ljbffr ...

lead the design, development, and optimization of our data infrastructure in a hybrid role based in Hounslow. You will build scalable pipelines with Spark, Hadoop, Hive, and AWS, maintain data warehouses on Oracle/SQL Server, and drive data quality and security. You will leverage Informatica and scripting ...

deployment of Large Language Models Experience building generative AI solutions Experience developing scalable machine learning systems Familiarity with big-data technologies such as Spark #CIBAppliedAI #J-18808-Ljbffr ...

understanding of statistics, machine learning methodologies, and model evaluation. Experience delivering AI solutions from proof-of-concept through to production. Exposure to Databricks, MLflow, Spark, or similar cloud-based data platforms. Ability to influence senior stakeholders and lead technical strategy Desirable Experience Experience building multi-agent AI systems ...

Python, C++; and Go. Deploy and manage applications using Kubernetes (K8s) and HPC/Grid Computing/Big Data technologies such as Slurm, LSF, Spark, Ray and Symphony in cloud environments. Required Qualifications, Capabilities, And Skills Formal training or certification on Software Engineering concepts and 5+ years applied experience. ...

person who knows what the data can and can’t support Nice to Have Experience in payments or with sensitive, regulated data Databricks or Spark experience alongside ClickHouse Experience as an early or founding member of a data team, including hiring This Might Not Be for You If... ...

diagnosing and fixing performance bottlenecks in live systems through profiling and measurement. Nice to Have: Experience with PostgreSQL/pgvector, OpenSearch (or Elasticsearch), Spark/Delta Lake, AWS, NoSQL databases, or Rust/C++ is useful. The Founders You'll partner with a founding team of AI PhDs ...

. Experience with Natural Language Processing (NLP) and Large Language Models (LLMs). Experience in areas of model development, data processing and streaming (Spark, Kafka), microservices in python (Flask or FastAPI), and interactive visualisations and User Interfaces (Streamlit, Plotly, Gradio etc). These lists are not exhaustive ...