What is a database engineer?

Database engineer: the bridge between the backend developer and the DBA.

Introduction

About 6 years ago I have been worked at a small company as a full stack developer. At this time, I had my first English written blog called ‘Morzel about Computer Programming’. In these days we have been fighting with some problems, that I was handling as the biggest challenges during my professional life. We got some deadlocks and a huge database – or at least I believed that was a huge database :D.

 

That was the time when I blogged about my database related challenges and one day, I got an offer at a rising startup. After some hesitation, I have accepted the job and I could start to work as a database engineer.

The first few months as a database engineer

The first thing that I had to learn is that I know almost nothing. My technical knowledge had huge holes and I had to fix the missing parts.

Parallel in learning the technical details, I had no choice, I learned a really complex domain model.

And now comes the key point:

The most important role of a database engineer

Is being a bridge between the backend engineers and the DBAs. I believe, there is nothing more important than this.

Components of a Database Engineer
Components of a Database Engineer

For me, I felt there were two groups who were always yelled at me and our team:

  • The backend developers wanted to implement their requests as soon as possible. Actually, we have blocked them because they had to use the stored procedures that we created.
  • The DBAs wanted us to implement everything in a best-optimized way: we had to make sure that we create perfect data structures and use them in a most efficient way.

It was really hard to meet both of these requirements. If we could finish our job fastly, the DBAs yelled us about the quality. When we have created the best quality code and data structures, the whole backend team was blocked because of us.

How to give value fast and have good quality as a database engineer?

After several attempts at refactoring our development flow, I found a way to meet all of the expectations: I started to work in multiple phases:

  • Implement interfaces: create an unoptimized data structure and give stored procedure skeletons to the backend teams.
  • Technical planning: plan the final data structure and basic usages.
  • Review the technical plan: validate the plan with the DBAs
  • Implementation: implement the final form

Using this flow, the backend team get an answer almost immediately. They will have their stored procedures and they can implement their code. In that time our procedures are doing nothing (or almost nothing) but they know their specifications. Is something like giving interfaces instead of implementations.

In the next step, we can figure out the perfect storage strategies. We can fine-tune indexes and do any other checks.

After creating a technical plan, we can ask the DBA team to review that. It is a very important step because we really do not want to fix the table or index structure in the live environment.

At this point, we got feedback both from the backend and the DBA team and we can implement the final form of the development.

Final thoughts

Being a database engineer is a strange job. To do this job in a proper way, you must have both database and domain skills and it is good to have some backend development knowledge too.

In that role, it is hard to work in a similar way to the other engineering jobs. A database engineer cannot just sit down, implement a whole feature and go forward to the next tasks. That role is more about communicating different teams and make sure every person gets that their want.

Learn by doing Coding Katas

For a long time, I have underrated coding katas. I used to say there is no reason to do the same task day by day just to improve my coding skills. Now, I learn a new technology stack, the Java stack and it has a lot of challenges for me.

After some learning stuff and fails, I have figured out that I spend too much time not by coding but by figuring out the code related infrastructure. Managing maven dependencies are a pain in the ass for me and almost every component has a new challenge because of the huge difference between .Net stack that I get used to and the Java stack that is a brand new to me.

So, after some fails, I have decided to do some coding katas just to learn the basic infrastructure and speed up myself.

FizzBuzz

FizzBuzz is a coding kata that teaches the basics of Test Driven Development. The task is simple. You count from 1 to 100 and for every number, there is the rule:

  • If it can be divided by 3, the output is ‘Fizz’
  • If it can be divided by 5, the output is ‘Buzz’
  • If it can be divided both by 3 and 5, the output is ‘FizzBuzz’
  • In other cases, we give back the input as an output

First day – OMG

The first day was terrible. First of all, I have created an empty Maven project. After that, I had to figure out what is the recommended project structure (that was the moment when I have realized that an empty Maven project does not contain the default project and package structure, I have to create this).

After these, I found the JUnit dependency and I thought that will be great. After some tests, I have figured out that the default JUnit version is 4 that I found and it does not support parameterized tests. After some googling, I met with some kind of plugin (or extension) that allowed me to write parameterized tests but in a very uncomfortable way (I wanted the same experience as I had using NUnit before).

And the problems came one by one. It takes about 5 minutes to implement a solution to the FizzBuzz problem. If I want to do that with a test-driven approach, it took about 20 minutes to me using .Net stack that I am familiar with.

On the first day, solving the FizzBuzz coding kata using Test Driven Development approach, took 58 minutes to me.

First reaction

I wanted to cry. I thought there is no way that such a simple exercise takes an hour for a Senior Software Engineer.

Analyze and improve

After a short walk, my mind calmed down and I could think about that hour. I tried to remember to the biggest gaps in my knowledge and wrote down a list of what I have to read during the night. That was on my list after the first day:

  • Maven project structure. Can I generate the default folder structure?
  • Unit testing. Is there any useful alternative to JUnit 4?
  • IntelliJ: code refactoring shortcuts
  • IntelliJ: run all tests shortcut

Doing the coding kata for a week

There was a long week when I felt pain every single day. I made a wrong Maven dependency structure, I have misconfigured the configuration file and could not run tests, I have tried 3 different unit testing tool and one time I made an unrealistic package structure.

But it took less amount of time to finish the Kata every single day.

Conclusion

After one week, I could finish the Kata under 20 minutes, including exception handling and boundary check.

Now, I can create a new project with necessary dependencies relative quickly and the most important part is that I have gained some confidence in using these tools.