Create New Account
Sign up to continue searching for suitable jobs in Web 3.0

OR
Terms of Use
Already have an account?

Log In to Your Account
Log in to continue searching for suitable jobs in Web 3.0

OR
Don’t have an account?
Uni Systems
Data Pipeline Engineer
about 2 hours ago | 3 views | Be the first one to apply

Data Pipeline Engineer

Full-time
Netherlands

About the company

Our UniQue IT people are the most valuable part of Uni Systems; their knowledge and experience has made us the leading and reliable systems integrator of today and has contributed to our steady financial growth. We have created and are maintaining a stable working environment for our employees, with countless opportunities to innovate and thrive. Our work culture recognizes our UniQue IT people, supports the free sharing of ideas and the flow of information via open communication, while appreciating and effectively utilizing the talents, skills and perspectives of each employee. At Uni Systems, we are providing equal employment opportunities and banning any form of discrimination on grounds of gender, religion, race, color, nationality, disability, social class, political beliefs, age, marital status, sexual orientation or any other characteristics.

Job Summary

What will you be bringing to the team?

šŸ“Setting up / improving pipelines to process all required documents that uniquely identify and trace decisions and processing steps. This is to be conducted on the provided classified sandbox environment, with provided performance hardware and toolsets. šŸ“Implementing / improving (missing) pipeline steps for marking duplicate files, based on file attributes, path (structure), and content (similarity), and establishing rules for determining if a file or structure is a duplicate. šŸ“Extracting document-format records from Functional Area Systems (FAS) databases and other performed backups. Archiving SMEs and system SMEs are available to guide target formats and interpret source system structure and data. Each FAS is processed individually; not all sprints address this item. šŸ“Processing / Monitoring various office, image, and video file types into accepted archiving formats, including metadata extraction and preparation of semantic indexes for search. šŸ“Automating the registration of all processed documents and their semantic indexes with the sandbox natural language search tool. šŸ“Automating the final transfer of all non-duplicate and extracted archive documents, including content and metadata, to the Institution archiving system. šŸ“Reporting the status, progress, and statistics of raw files being converted into archive formats, along with associated metadata and search indexes. šŸ“Delivering full reporting of results, pipeline step traceability, and documented (stakeholder-approved) exceptions.

Requirements

šŸ“What do you need to succeed in this position? šŸ“Master’s degree in Computer Science, Engineering, or a relevant field (an advanced degree in Data Science is preferred) šŸ“At least 3 years of practical experience in the field of data science and/or data analytics šŸ“Experience using data processing, visualization, and analytics software packages and development environments, preferably such as KNIME, VS Code, GitLab, Power BI, Jupyter Lab, and Docker-based APIs šŸ“Experience with Big Data processing, creating and utilizing containerized building blocks, and running containers (APIs) on Kubernetes clusters šŸ“Proficient in programming/scripting languages such as Python, R, and SQL, and working with data formats like CSV, XML, and JSON

If this role isn’t the perfect fit, there are plenty of exciting opportunities in blockchain technology, cryptocurrency startups, and remote crypto jobs to explore. Check them on our Jobs Board.

Salaries for similar jobs:

Similar jobs

about 2 hours ago | 4 views | Be the first one to apply
about 2 hours ago | 3 views | Be the first one to apply
$100,000 To $150,000 per year
about 3 hours ago | 3 views | Be the first one to apply
Full-time
London, United Kingdom
$100,000 To $120,000 per year
3 days ago | 66 views | 5 applications
Full-time
Remote
$84,000 To $110,000 per year
3 days ago | 54 views | 2 applications
Full-time
Remote
$80,000 To $95,000 per year