รายละเอียดงาน

• - Create pipeline to ingest data from SFTP, GCP, BigQuery, On-premise database like Oracle, MySQL in Databricks using python and PySpark - Working with Hadoop and Spark-Submit.
• - Create Spark-Submit pipeline and maintain the legacy pipeline
• - Develop new pipeline for migration pipeline from Scala and Spark-Submit to PySpark in Databricks and create CI/CD in GitHub
• - Migrate HADOOP that use Spark-Submit pipeline from old server to new one by recreate same environment and Scala and Java library
• - Manage workspace, monitoring usage, managing workspaces resource, manage control access, service account access and manage unity catalog. Also develop Databricks audit log - Manage and maintenance the development cost and customize the system to match to KPI cost
• - Manage team resource with Agile Process - ETC. such as maintain pipeline in Databricks, Sparks-Submit

รายละเอียดงาน

• - Design Data pipeline for creating Data Analyst report in Power BI use python which connect to Azure SQL Database, MSSQL Server, MySQL
• - Create Automation flow use MS Power Automate to send automate email, import answers from MS Form to Excel in MS OneDrive. And use python to get data from MS Excel in MS OneDrive/SharePoint to import to Azure SQL Database - Implement SQL script work process, DDL processes
• - Create Cron Job in Azure Function Service by developing python script for importing data from Data source to Azure SQL Database

รายละเอียดงาน

• - Serve as a consultant for a global company in the role of Data Engineer, focusing on creating efficient data pipelines and performing ETL processes, while utilizing Teradata and PowerCenter Informatica for data analysis.
• - Develop and implement stored procedures and ETL processes using SQL scripts within an Oracle database to ensure data integrity and accessibility.
• - Implement SAP Business Objects reporting solutions and create deployment packages for reports, ensuring they meet business requirements and are delivered on time.

รายละเอียดงาน

• - Create data pipeline, cleansing data and import to Google Cloud BigQuery use Python (pandas, NumPy, PySpark) for collecting and processing data. And update data from on-premise database (MySQL, MariaDB, MongoDB) to BigQuery use task scheduler
• - Create streaming data pipeline use Pub/Sub (Google Cloud) by Python - Create web scraping, cleansing, processing and insert to database (MySQL, MariaDB, MongoDB, BigQuery) use Python
• - Create BI visualization use Power BI and connect data from 3rd party data source, Google Cloud BigQuery and embedded report in web application

รายละเอียดงาน

• - ETL use Oracle Data Integrator 12c
• - Use Cleaned data to create BI data model by Oracle Administrator tool, Tableau
• - Create Visualization from data model use OBIEE 12c and visualization tool use Oracle Data Visualizations and Tableau Desktop and publish to Tableau Public for embedded report in organization website
• - Manage Tableau Server
• - Create query script in Oracle Database 12c
• - Implement website use HTML5/CSS, PHP (CodeIgniter, Yii, Laravel framework)
• - Others in responsibilities such as TOR Documentation

รายละเอียดงาน

• - Conduct comprehensive testing on B2B projects using both Manual and Automated Testing methods, leveraging tools like Robot Framework and Selenium with Python for effective results.
• - Perform User Acceptance Testing (UAT) on B2B projects, executing thorough Manual Testing before and after the production phase to ensure a smooth Go-Live experience.
• - Develop detailed test cases to outline testing scenarios and expected outcomes, ensuring robust coverage of all functional requirements.
• - Execute Manual Testing during System Integration Testing (SIT) and in the production environment, identifying any issues and ensuring quality standards are met.