Meet Charlotte…
My current job title is Lead Data Engineer, but this captures a variety of activities. The things I’m responsible for range from technical leadership of a project team (coordinating the Development Team and contributing to the technical work needed to deliver the Quantexa), to leading the Data Practice (Quantexa's community of junior Developers within the Delivery function), and acting as a key Delivery stakeholder during the rollout of new features across Delivery (coordinating engagement between Delivery projects and R&D).
I’ve been a Lead Data Engineer since 2022 but have been a Data Engineer since joining Quantexa as a graduate in the summer of 2018.
What did you want to be when you were little?
I watched a lot of CSI, so until my teenage years I was adamant I wanted to be a Forensic Scientist. Once I’d grown out of that, Maths was my favourite subject at school and I was good at it, so my focus shifted to studying Maths at university and perhaps going into academia, but no real plan for a career. I’m the first generation in my family to go to university so I didn’t have a great idea of what jobs were out there until years later!
What first sparked your interest in pursuing a career as a Data Engineer?
The shift from Maths to Computer Science (and then into Data Engineering) happened at university. I had my heart set on going to the University of Warwick and got a place on their Discrete Mathematics course – a fun name for a Maths and Computer Science joint honours. During my time at Warwick, I realised that university (and academic) Maths wasn’t my calling, so I started to pick up more Computer Science modules and found it something I really enjoyed.
When it came to job hunting, initially, due to severe imposter syndrome (which we’ll get into later), I didn’t think I’d be able to get a job in tech, so I was originally looking for business administration or more generic consultancy roles. On speaking to the head of undergrad in my department about this, he pushed me to attend one of the tech careers fairs and by complete chance, Quantexa was attending this event. I liked the sound of what they did, which included the still-famous Michael Greene demo so I decided to take a chance and apply. The rest is history!
How do you continue to grow and develop in your role?
When I started at Quantexa, I knew that I had lots to learn. Going from a degree and having (limited) theoretical knowledge to applying that knowledge to real-world problems was a huge leap. The first thing that helped with this was going through the Quantexa Academy.
After that, the main source of growth and development in the first few years was through hands-on experience on projects. I moved projects consistently every 6 months, working with different people each time. This meant I was learning new technologies, new Quantexa features, new use cases, and I was always learning from new people, each with their own background and experience. This was invaluable in learning how to deliver a Quantexa project and lead a Technical Development Team – I owe a lot of my management style and ability to the Technical Leads (TLs) I worked under during those years.
Once I became a TL and gained more managerial responsibility, I was offered the opportunity to take a management training course. This was in combination with being invited to join the Tech Lead Practice - a community of Tech Leads within Quantexa where we share our experiences and work together to improve and streamline the role of a TL and the delivery of Quantexa projects. This meant that even as I was becoming a TL, I was equipped with the theoretical knowledge and skills needed, and surrounded by people to draw wisdom from, both of which helped to make the transition into more of a management role pretty seamless.
Can you describe your career journey?
I joined Quantexa 6 days after my graduation (to be specific) and went straight onto the Data Engineering Academy, which was still being created. Once completing this in around October 2018, I was assigned to my first project. The step up from the Academy to project work was a big change, going from learning the basics to delivering a complex solution to a Tier 1 bank was a huge learning curve for me. By the time I rolled off of that project in March 2019, I felt I’d been through the Academy all over again with the amount I learned being surrounded by a team of Quantexans who had already been with Quantexa for months if not years.
I then worked on a few different projects as a Data Engineer, until November 2019 when I was asked to do a secondment to Australia for 3 months to help with two new projects.
This was a big challenge!
The first project in particular was one to remember, it included setting up a project environment from scratch for the first time, a 12-hour time zone difference away from the support of the London office. Nonetheless, one I really enjoyed (the weather definitely helped). I then headed to Melbourne to join another project, until March 2020 when the proof of concept (POC) wrapped up, at which point the team headed back to the UK (just in time for lockdown).
During 2020, I was given my first leadership opportunity. This started with a “Sub-Team Lead” position which I held for seven months, I then moved into my first official TL role. 2020 was the year where my role changed the most, with the shift from Data Engineer to Team Lead. This came with challenges as everyone was working remotely, but again a great opportunity and I learned a lot about how to manage and effectively lead a team. Around this time, I took the management training course to learn the theory behind some of the actions I was putting into practice.
Since then, I’ve led three further projects, as well as technical support and oversight on three others. It’s been a great balance of seeing a project through from start to finish (including attending awards dinners for the work we did) and jumping into a project for short bursts when help is needed which has kept me on my toes.
While the projects I've been on have largely molded my career progression and experience historically, as I’ve become more senior, my responsibilities and workload have evolved. As I mentioned in my intro, I’m now involved in various internal workstreams and work a lot more across the Delivery function. This means I’m directly influencing the direction we take as a department, and the roadmap and development undertaken by R&D ensuring we serve our client’s needs. This has been an amazing new challenge and helped me change my perspective to be more aware of the business context of the projects I lead.
What are some of the biggest challenges you faced in becoming a Lead Data Engineer?
As mentioned, when starting my career I had a huge case of imposter syndrome. I’d ended up on the Maths and Computer Science course and throughout was surrounded by people who spent their spare time coding for fun whereas I'd only ever written code for an assignment and was otherwise unaware of the tech world.
This meant I didn’t feel very qualified for a job but was encouraged to apply by university staff anyway. Unsurprisingly that feeling didn’t go away once I'd started at Quantexa. The Academy at the time was a pretty light touch, Quantexa was just over 2 years old, so a lot of the job was having to learn and pick things up as I went – a terrifying prospect when I felt I’d swindled my way into the job in the first place!
I’m not going to say it’s been easy to overcome the feeling of imposter syndroms, and it's been a slow process of gaining confidence over time, but I’m glad to say I feel it less and less often these days (though I still have my moments).
One thing that I found helped was to stay in touch with my peers who joined around the same time as me, as we were all going through the same experience. Knowing that other people were feeling the same way reassured me that finding it difficult didn’t mean that I wasn’t cut out for the job.
This is still relevant today, I've stayed in contact with the same group as we became TLs, mentors, and took on new responsibilities. They've been a source of reassurance whenever I’ve started to feel I’m in over my head. Knowing that all the nervousness and anxiety are actually pretty common and that it’s ok to be figuring things out as you go along.
If there are areas I feel I’m not confident in, I try to identify low-risk ways to gain that confidence.
A specific example of this for me was with Entity Resolution (ER) – a core Quantexa concept. There was a point where I hadn’t done a huge amount of ER development or Tuning and felt it was a topic I didn’t know enough about. To gain some confidence, I signed up to present a session to the graduate scheme on ER. The stakes were relatively low – I was presenting key concepts to a group of people who were still learning the ropes (and were probably feeling some imposter syndrome themselves). It allowed me to reassure myself that, I do know how this works and to consolidate (and expand) my understanding – teaching others is a great way of making sure you yourself fully understand the concepts you’re presenting.
Lastly, I've found engaging with the feedback process a great way to overcome obstacles. If you're feeling like an imposter, it can be daunting to actively reach out for feedback, but often the feedback you receive will be instrumental in getting past that mental block. Either it will be positive feedback, reassuring you that your imposter feeling is just that, a feeling, or it's constructive feedback on how to improve. Having clear actions and goals in mind taken from this constructive feedback, is a great way to gain confidence and explicitly mark your progress which can lead to moving past that imposter feeling.
What key skills and personal traits are essential for succeeding in your role as a Lead Data Engineer?
Most importantly (but perhaps least excitingly), it’s important to be able to manage your time and priorities. Having a clear view of the most urgent and important task on your to-do list makes for a less stressful time as it prevents things from slipping through the cracks or being left until the last minute.
Often it's relatively clear what this priority should be, but when managing workloads across different projects or workstreams, it can require a lot of discipline to stand your ground. When faced with a choice between chipping away at a longer-term piece of work (that will eventually have a high impact across all of Delivery) vs. fielding short-term urgent client requests or asks from your team, it can be difficult to push back but it's important to trust your judgment and say no when needed.
Relatedly, to make the above remotely possible, I also find the art of delegation and coaching to be critical in making sure things run smoothly. We all only have 8 working hours in a day, and as much as I’d love to have my superhero cape on and singlehandedly solve all of the problems, in the long term it’s better for everyone (and less stressful) to invest time in the people around you and make sure they feel sufficiently equipped with the skills to step up and take over when their time comes.
What do you find most rewarding about your job as a Lead Data Engineer?
There are two things for me.
- The first is the reactions from clients when they first see their data in the Quantexa Platform. I’ve worked on projects where Quantexa was engaged to visualise data for the first time. Up to that point, the data had been static tables in databases – so being able to bring that data to life in the Quantexa UI, dynamically building networks, visualising risk, and offering a whole new perspective - the reaction never gets old!
- The second is the opportunity to work with more junior members of Delivery and take an active hand in their development. This has ranged from being a mentor- being directly responsible for my mentee’s career progression, to being the manager of a project team and upskilling the members.
A particularly nice example of this came earlier this year. I worked closely with a team member over the course of my previous project. He joined the team as one of his first Quantexa projects so spent time upskilling and gaining Quantexa experience until eventually, in January I was able to fully hand over the project to him as the TL – a real full-circle moment.
What one piece of advice would you give to someone aspiring to become a Lead Data Engineer?
In short – be curious.
I’ve found being interested in the things going on around you is a great way to start to understand the broader context of your role – asking questions, keeping up with project details and how the client relationship is going, chatting with people on other projects to see what new features they're implementing etc. – all of this will help you make those prioritisation calls, and make better use of your time.
As a curious person, this comes naturally to me- I love to hear the details of other people’s projects, what's going on and why, and coming back to Community support posts that sound particularly interesting or potentially relevant to see what the solution was in the end – just in case it might be useful in the future.
I’ve found that experience, information and context are all incredibly vital in the role of a TL / Lead Data Engineer and the best way to gather this is through keeping up with what’s going on around you as much as possible.
