Daniel Cerqueira

Background

About Daniel Cerqueira

I’m a data engineer with a bachelor’s degree in Psychology and a background in research and data analysis. My final paper was a systematic review that measured the quality of the scientific method in studies that used mindfulness as the primary strategy to improve the sample’s psychopathology symptoms in the therapy environment. In college I was part of a research group that made studies about human behavior and its cognitions, doing correlations and inferential tests using IBM SPSS and Excel. This experience made me interested in understanding how raw data can be transformed and organized to have a better understanding about human behavior and the variables that affect it. These insights are valuable for companies to make better use of them.To improve my data analysis abilities, I self-learned Python, its libraries and frameworks (Pandas, NumPy, Matplotlib), and SQL (MySQL and PostgreSQL), throughout the graduation, doing freelance work in the past few years. However, as I was gaining experience and learning more about the world of data, I developed a passion about data engineering. Understanding how to extract data from different sources, cleaning it to improve its qualities, and loading its contents in a data warehouse or data lake, became a priority of mine. Through the process, I learned how to use various tools, such as Apache Hadoop, Apache Spark with the PySpark API, Apache Hive, Apache HBase, and doing the orchestration using Apache Airflow, automating the ETL/ELT process. Also, I learned how to use the tools that connects with the previous ones to deploy, store and manipulate the data on the cloud, such as AWS Redshift, Snowflake, AWS EMR, Databricks, AWS EC2, Google BigQuery. The world of data is an ever changing field. New tools are being developed to fill the gaps of the previous, with the goal to make better use of the data available. Being up to date with the new technologies emerging is an important asset for a data engineer to have. In my free time I like to understand the new trends, extracting value from different types of data, how to apply them to the business, and improve the abilities I already have.