Database engineer: the bridge between the backend developer and the DBA.
Introduction
About 6 years ago I have been worked at a small company as a full stack developer. At this time, I had my first English written blog called ‘Morzel about Computer Programming’. In these days we have been fighting with some problems, that I was handling as the biggest challenges during my professional life. We got some deadlocks and a huge database – or at least I believed that was a huge database :D.
That was the time when I blogged about my database related challenges and one day, I got an offer at a rising startup. After some hesitation, I have accepted the job and I could start to work as a database engineer.
The first few months as a database engineer
The first thing that I had to learn is that I know almost nothing. My technical knowledge had huge holes and I had to fix the missing parts.
Parallel in learning the technical details, I had no choice, I learned a really complex domain model.
And now comes the key point:
The most important role of a database engineer
Is being a bridge between the backend engineers and the DBAs. I believe, there is nothing more important than this.
For me, I felt there were two groups who were always yelled at me and our team:
- The backend developers wanted to implement their requests as soon as possible. Actually, we have blocked them because they had to use the stored procedures that we created.
- The DBAs wanted us to implement everything in a best-optimized way: we had to make sure that we create perfect data structures and use them in a most efficient way.
It was really hard to meet both of these requirements. If we could finish our job fastly, the DBAs yelled us about the quality. When we have created the best quality code and data structures, the whole backend team was blocked because of us.
How to give value fast and have good quality as a database engineer?
After several attempts at refactoring our development flow, I found a way to meet all of the expectations: I started to work in multiple phases:
- Implement interfaces: create an unoptimized data structure and give stored procedure skeletons to the backend teams.
- Technical planning: plan the final data structure and basic usages.
- Review the technical plan: validate the plan with the DBAs
- Implementation: implement the final form
Using this flow, the backend team get an answer almost immediately. They will have their stored procedures and they can implement their code. In that time our procedures are doing nothing (or almost nothing) but they know their specifications. Is something like giving interfaces instead of implementations.
In the next step, we can figure out the perfect storage strategies. We can fine-tune indexes and do any other checks.
After creating a technical plan, we can ask the DBA team to review that. It is a very important step because we really do not want to fix the table or index structure in the live environment.
At this point, we got feedback both from the backend and the DBA team and we can implement the final form of the development.
Final thoughts
Being a database engineer is a strange job. To do this job in a proper way, you must have both database and domain skills and it is good to have some backend development knowledge too.
In that role, it is hard to work in a similar way to the other engineering jobs. A database engineer cannot just sit down, implement a whole feature and go forward to the next tasks. That role is more about communicating different teams and make sure every person gets that their want.