1 to 25 of 167 Big Data Jobs

NO SPONSORSHIP DATA ARCHITECTURE SALARY: $161K - $200K plus 12% bonus LOCATION: Remote except in : California, Delaware, District of Columbia, Kentucky, New Hampshire, New Jersey, New York, Pennsylvania, Vermont, West Virgina You will be developing data architecture plans for cloud data transformation, data governance, data quality … data standards. CRM MDM generative AI and data science Microsoft azure Real Time integration ETL data warehousing data catalog analytics BI operation datastores azure data bricks azure AD azure ML cloud era Spark sqoop Hive HDFS flume storm and kafka This role will be responsible … for developing data architecture plans and driving the plans to fruition in collaboration with business and IT. You will play a key role in driving a number of data and analytics initiatives including: cloud data transformation, data governance, data quality, data standards, CRM, MDM more »

in: California, Delaware, District of Columbia, Kentucky, New Hampshire, New Jersey, New York, Pennsylvania, Vermont, West Virgina* A prestigious company is looking for a Data Architect. This architect will focus on enterprise data architecture which includes cloud data transformation, data governance, data quality, data standards, CRM, MDM, Generative AI and data science. Responsibilities: This role will be responsible for developing data architecture plans and driving the plans to fruition in collaboration with business and IT. Play a key role in driving a number of data and analytics initiatives including: cloud … data transformation, data governance, data quality, data standards, CRM, MDM, Generative AI and data science. Define cloud reference architectures to promote reusable patterns and promote best practices for data integration and consumption. Guide the data science team in implementing data models and more »

We are unable to sponsor for this permanent Full time role* *Position is bonus eligible* Prestigious Enterprise Company is currently seeking an Enterprise Data Architect with Azure Cloud and MDM experience. Candidate will be responsible for developing data architecture plans and driving the plans to fruition in collaboration … with business and IT. You will play a key role in driving a number of data and analytics initiatives including: cloud data transformation, data governance, data quality, data standards, CRM, MDM, Generative AI and data science. Candidate will define cloud reference architectures to promote … reusable patterns and promote best practices for data integration and consumption, as well as guide the data science team in implementing data models and analytics models. You will also serve as a data science architect delivering the technology and architecture services to the data science more »

Big Data Engineer - Fully Remote in Africa/EU As a Big Data Engineer you will have a strong understanding of big data technologies with an exceptional ability to code. You'll be providing technical leadership, working closely with the wider team and focusing … Code, TDD, BDD, continuous integration, continuous delivery, and DevOp s You will, among other tasks: Define, design, and develop services and solutions around large data ingestion, storage, and management such as withRDBMS, No SQL DBs, Log Files, Events. Define, design, and run robust data pipelines/batch jobs … Redis etc. Mocking (any of the following Mockito, ScalaTest Spock, Jasmine, Mocha). Build tools (One of SBT, Gradle, Maven). Knowledge of AWS Big Data/Analytics services - S3, EMR, Glue, Redshift, QuickSight, Kinesis. Desierables Skills Experience of big data environments (also advising best practices more »

Data Engineer Our client is a highly profitable and ambitious financial service business that has grown from scratch to c.£400m annual sales and over 600 head office staff, servicing over £30bn of assets under management and 20% of the UK financial adviser marker in approx. 17 years. Its … is on the next stage of its phenomenal journey and were now looking to help further strengthen the team, by appointing a first-rate Data Engineer. Reporting to the Head of Data, you will play a key team role in building a scalable, reliable and high impact solution … which helps the business become more data driven and deliver actionable insight. Using modern data engineering tools and technologies, y our key responsibilities as Data Engineer will include: Evaluating business needs and objectives - working with Data Analysts and other users to understand the needs for data more »

Enterprise Data Architect Salary: Open + bonus Location: Remote *We are unable to provide sponsorship for this role* Qualifications 12+ years of relevant experience Strong cloud data architecture knowledge (preference for Microsoft Azure) Experience developing architecture strategies and plans to enable cloud data transformation, MDM, data governance, and data science capabilities. Design reusable data architecture and best practices to support batch/streaming ingestion, efficient batch, Real Time, and near Real Time integration/ETL, integrating quality rules, and structuring data for analytic consumption by end uses. Ability to lead software evaluations … including RFP development, capabilities assessment, formal scoring models, and delivery of executive presentations supporting a final recommendation. Well versed in the Data domains (Data Warehousing, Data Governance, MDM, Data Quality, Data Standards, Data Catalog, Analytics, BI, Operational Data Store, Metadata, Unstructured Data more »

Big DATA Engineer - 6 months + ext. - Hybrid/Fulda.* Job Description: Must Have: * 5+ years bigdata Spark programming skills * Experience in building complex data pipelines using Spark and especially PySpark on top of cloudera distribution. * Experience in performance tuning Spark application. * Experience in scheduling and running … Spark applications. * Hands-on experience with fundamental Hadoop tools and technologies Good to Have: * Experience in Palantir platform as a data engineer or knowledge of Palantir platform is added plus * Conceptualize and design big data system involving Hadoop, PySpark, Hive * Design functional and technical architectures. * Develop applications … on the big data platform using open-source programming languages. * Work closely with Administrator, Architect and Developers * Knowledge on Cluster management and storage mechanism in Big Data Cloud Essential Skills: * Strong analytical skills * Strong framework design skills * Strong code standardization skills If you are interested please more »

Main Purpose of the Role: Develop and maintain data analytics solutions, including data pipelines and reporting, leveraging our data lake (Big Data) and relational databases. Monitor database jobs and suggest improvements to optimize performance and efficiency. Liaise effectively with both business and development tea ms … to ensure alignment of data solutions with business objectives. Key Responsibilities ETL/ELT Development (Extract, Transform, Load): Hand-code data integration processes, with experience in tools like Apache NiFi being a plus. Big Data Technology Stack: Utilize a strong command of Big Data … Platform (BigQuery). Responsibilities include job monitoring (eg, Ambari), performance tuning, and debugging execution plans. Query Languages: Proficiency in HiveQL or relational SQL for data manipulation and query execution. Scripting: Develop scripts using languages such as BASH, Python, and VBA to automate and streamline data processes. BI Reporting more »

Overview: Step into an exciting role where you'll contribute to the advancement of our state-of-the-art data lake platform, leveraging Big Data technology. Our mission is to streamline current processes and innovate new ones, driving excellence and efficiency. You will join a dynamic team … unique expertise. As a Back End developer within our "Application Development for Cash Datawarehouse" unit, you'll manage applications that store and process trading data from various stock exchanges, including a prominent European exchange. Your work will also involve data analysis and interactions with internal stakeholders. No need … new role and fully realize your potential. Key Responsibilities: Develop using Spark/Scala Read inputs from HDFS, HBase/Phoenix, Kafka Analyze trading data, such as surveillance alerts and Market Maker ratings Maintain batch flows with Control-M Create data queries in Zeppelin Qualifications: Experience with Big more »

rate: 200-300€ per day Overview: Step into an exciting role where you'll contribute to the advancement of our state-of-the-art data lake platform, leveraging Big Data technology. Our mission is to streamline current processes and innovate new ones, driving excellence and efficiency. You … unique expertise. As a Back End developer within our "Application Development for Cash Datawarehouse" unit, you'll manage applications that store and process trading data from various stock exchanges, including a prominent European exchange. Your work will also involve data analysis and interactions with internal stakeholders. No need … new role and fully realize your potential. Key Responsibilities: Develop using Spark/Scala Read inputs from HDFS, HBase/Phoenix, Kafka Analyze trading data, such as surveillance alerts and Market Maker ratings Maintain batch flows with Control-M Create data queries in Zeppelin Qualifications: Experience with Big more »

Digital Intelligence is home to 4,500 digital, cyber and intelligence experts. We work collaboratively across 10 countries to collect, connect and understand complex data, so that governments, nation states, armed forces and commercial businesses can unlock digital advantage in the most demanding environments. We are looking for experienced … Data Engineers to join our team following continuous growth and success. As a Data Engineer you will be involved in: Design, develop, test and support data collection, data integration and ETL applications to make information and data available to key client stakeholders and technical interfaces … Model data requirements, data sources and data flows to bring order and structure to programmes of work Defining how and where data is created, mastered and destroyed to ensure proper control over the lifecycle of corporate data assets Using categories of products that can be more »

Data Product Specialist Job Type: Full-time Location: Bristol Salary: £42,000 - £46,000 Work Pattern: Hybrid (2 days onsite) We are seeking a highly skilled Data Product Specialist for our Bristol-based client. This role involves supporting, maintaining, and enhancing their data platform to ensure it … aligns with industry best practices and meets the organisation's needs. The ideal candidate will have a robust understanding of big data and analytics, data engineering, and data design, along with experience in cloud integration and data security. Day to Day of the role: Optimise … and manage the data infrastructure, including DataBricks, Power Platform, Azure, and legacy data environments. Ensure efficient data processing and real-time analytics capabilities. Implement data ingestion frameworks, stream processing, and cloud-native data services. Maintain data security best practices, compliance requirements, and governance frameworks. more »

site Enhanced DV Clearance to start - must be active £685p/d inside ir35 Contact until March 2024 - high chance of extension Expertise in Data, Java/Python, Hadoop, ETL Who are we? We are recruiting a Data Scientist with Enhanced DV clearance to work with an industry … for the best talent to join us as we continue to bring the best to the table. We work alongside our customers using modern data engineering/science methods and the latest technologies. What will you be doing? Play a pivotal role in analysing complex data sets, developing … predictive models, and implementing data-driven solutions. Develop and implement predictive models and machine learning algorithms. Utilise Java or Python for data analysis, model development, and automation. Leverage Hadoop and other big data technologies to manage and process large data sets. Apply software engineering best more »

Digital Intelligence is home to 4,500 digital, cyber and intelligence experts. We work collaboratively across 10 countries to collect, connect and understand complex data, so that governments, nation states, armed forces and commercial businesses can unlock digital advantage in the most demanding environments. We are looking for experienced … Data Engineers to join our team following continuous growth and success in the UK Government sector. Our people are what differentiates us, they are resourceful, innovative and dedicated. We have a mix of generalists and specialists and recognise that this diversity contributes to our success. We recognise the benefits … diverse opportunities for our people to develop their careers in new areas of expertise and with new clients. Our projects are spread across the Data Science lifecycle from delivering Research and Proof of Concept models, to building and delivering operational solutions. Machine Learning (ML) Engineers are responsible for designing more »

position, you will plan and direct business unit operations and the work of a team who produce advanced analytics algorithms, AI techniques and innovative data science solutions, AI-enabled automation, and predictive modeling to drive the success of strategy implementation. You will ensure team members are knowledgeable in data mining and data analysis methods, adept with large data science, Artificial Intelligence, causal AI and Generative AI techniques, computational programing capabilities, practical problem-solving skills, and possess the ability to articulate solutions to non-technical consumers or partners. As the director, you will develop partnerships across the … data engineering and data management teams, and external or created data sources to apply data mining techniques in preparation for analysis or use of enterprise data assets. THE IMPACT YOU WILL MAKE The Director of Data Science & Artificial Intelligence (AI) role will offer you more »

MCS Group is delighted to be partnering with a local financial services organisation as they add a Data Architect to their team in a role that will see you have real impact. The position will see you working on the company wide digital transformation and change projects. You will … work with a dedicated team of data scientists and engineers to drive data strategy and have the opportunity to lead in an organisation that values growth and flexibility. The package: Salary £70,000 to £80,000 depending on experience Car allowance Health insurance 10% Company Pension contribution Market … days) Flexible, Hybrid working and training opportunities. About the role: This is an exciting time to join and be involved in shaping the data landscape within the organisation. You will have the opportunity to work on collaborative Greenfield Data Projects. You will contribute to the full data more »

Lead Data Engineer Chichester St, Belfast BT1, UK Full-time Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies and innovation collide, we create opportunity from complexity. Our diverse teams of experts combine innovative thinking and breakthrough use of technologies … the team will look to you for support, guidance and help with decisions. You'll have: Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Perform tasks such as writing scripts … extracting data using APIs, writing SQL queries etc. Work closely with other engineering teams to integrate data engineering component into production systems. Knowledge of data cleaning, wrangling, visualization and reporting, with an understanding of the best, most efficient use of associated tools and applications to complete these more »

The Role: This role is pivotal in ensuring data solutions are scalable, sustainable, and aligned with both local and enterprise-wide architectural strategies, particularly in cloud migration. In this role, you will collaborate closely with Group Architects, the Head of Data & Analytics, and technology leads to craft a … data architecture roadmap that supports new technology platforms, advanced analytics, and AI initiatives. You will guide cross-functional teams, ensure data quality, and implement effective data management tools and practices. Additionally, you will communicate complex technical information to a broad range of stakeholders, establishing yourself as a … to integrate new technologies and policies effectively. This role is designed for a high-impact leader keen to play a key role in influencing data-driven strategies and enhancing technological capabilities. Key Responsibilities: Lead the design, development, and maintenance of data architecture solutions. Collaborate with key stakeholders to more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

Big Data Developer/Scala Developer Hybrid role, very flexible. (Must be based in Czechia) Rates: Open/Negotiable Overview : Step into an exciting role where you'll contribute to the advancement of our state-of-the-art data lake platform, leveraging Big Data technology. … unique expertise. As a Back End developer within our "Application Development for Cash Datawarehouse" unit, you'll manage applications that store and process trading data from various stock exchanges, including a prominent European exchange. Your work will also involve data analysis and interactions with internal stakeholders. No need … new role and fully realize your potential. Key Responsibilities: Develop using Spark/Scala Read inputs from HDFS, HBase/Phoenix, Kafka Analyze trading data, such as surveillance alerts and Market Maker ratings Maintain batch flows with Control-M Create data queries in Zeppelin Qualifications: Experience with Big more »

Data Migration Architect (Holds Active SC) Contract Remote Outside IR35 The data management industry is evolving rapidly, driven by the increasing volume of data generated daily and the growing need for efficient data storage, processing, and analysis. With the rise of cloud technologies and big data analytics, businesses are seeking robust data architectures to streamline operations and gain strategic insights. Our client stands at the forefront of this transformation, leveraging the latest advancements in data warehousing, lake houses, and cloud platforms. We are seeking a highly skilled Data Migration Architect … to join our client’s dynamic team. The successful candidate will play a pivotal role in designing and implementing data migration strategies, ensuring seamless integration across various platforms and systems. You will be required to have Public Sector experience working and supporting multi-disciplinary Agile teams. Key Responsibilities: Design more »