Where better than to check the internet for inspiration, especially if you need some interview questions.
The following interview questions are from an article taken from a large job website, regarding tips for Automation Test Engineer interviews (I’ve mainly been involved in hiring Quality Assurance experts). We’ll go through them one by one together, and I’ll explain what I think about the question. It’s important to remember that a good interview question is subjective. What I think is a good question you may not think is a great question. And I’m by no means an expert on conducting interviews, but I have been involved in hiring new team members and asked a lot of questions. So it seems like an article I could give my take on.
For me, a good interview question is one where the interviewee opens up and can share a lot of their opinions on a subject. The answers are based on my experiences interviewing for Senior level Automation Engineers. But these questions and answers can be applied to any level of experience — especially if you are claiming on your CV to be an experienced professional.
What do you think are some advantages of automation testing?
This is a great question. Questions that start with “what” are my favourite types of interview questions. “What is your favourite, or what is not your favourite and why”. If anyone has been in a job for +5 years then hopefully they’ve learnt a lot and can answer questions related to their work. For example, for an Automation Test Engineer a good question might be “Do you have a favourite test case creation approach (eg TDD or BDD). Or is there something else? And why is this your favourite and why are the others not your favourite?”. It is so open-ended which makes it incredibly easy to start rambling. The beauty in the question is that there is no right answer. I’m not looking for a particular answer anyway. How they structure their answer is important.
What kinds of frameworks have you worked with?
A straightforward question, with no real right or wrong answer. A framework is a way of creating tests— Data Driven, Keyword Driven, Behaviour Driven. It’s not too tricky and the question allows for some critical thinking about frameworks and can encourage discussion. An interviewee can give anecdotal examples of frameworks they have used. Did it provide for easy test case creation? How readable was it from a software engineering point of view? Any issues with maintenance? But it seems a bit too obvious and like this question gets asked a lot. I’ve been asked this question as an interviewee.
How many test cases have you automated in a day?
This. Is. A. Horrible. Question. Questions like this should no longer be in QA interviews, at any experience level. They tell you very little about an interviewee. The question is based on a useless, ambiguous metric. One of the problems is that there are so many questions within this question — what is your definition of “a test case” (it could be just opening a page and clicking on a link), what are you using to create Automation Tests (something easy like Selenium IDE or something more involved like mocha/chai/supertest/typescript?), how long is “a day” (24 hours, 12 hours, 8 hours, 3 hours?). I have been asked this by an interviewee too — “how many test cases am I expected to perform in one day?”. IIRC our answer to the interviewee was “we don’t encourage QAs to think like that”.
An additional note related to nonsensical information in CVs: putting percentages in job roles. For example, created automation tests and reduced bugs found by 90%. It seems strange to put such a claim in a document that is at best only read for 2 minutes before saying yes or no. It makes me wonder how was that magic number even calculated out anyway? Are you telling me that you had identical conditions over multiple different cycles of work where you could gather 100% reliable metrics to determine that your tests alone reduced bugs by 90%?! It’s absolute garbage. And if I do see this in a CV, I make sure to bring it up in a face-to-face interview. And if you’re honest and tell me that this happened because 9 of the 10 QAs didn’t contribute to the Sprint (so there were 90% fewer bugs found…), then that is fine. I’ll ask questions like how would you make this a controlled environment to get accurate metrics for Automated or something like this.
How familiar are you with Selenium?
There’s something about this question that I don’t like. Sure, if you’ve got a candidate who’s interviewing for a job creating Selenium tests then this is great. But if you’re not, then why ask this? If Selenium wasn’t mentioned in the job description and I got asked this in an interview, I would ask back with a statement that sounds like a question “the job description doesn’t mention knowledge of Selenium” and then make a mental note that this job isn’t for me and that they will have me doing some random tasks that are not in the JD. Or I would answer something about UI tests using a JSON wire protocol are arguably more flaky than using a UI test framework that controls a browser window.
How would you select a test case for automation?
I know this is a popular question. I don’t love it and I don’t hate it. It’s a good question, that promotes discussion and opinions. It allows the interviewee to demonstrate critical reasoning about the merits of a test case and what makes it suitable for Automation. It’s just not a question that is my repertoire of interview questions as it seems a bit like a question that would probably get asked a lot. I still ask it though, but more as a warm-up question. Or I’ve asked it to more experienced Manual QAs/
How do you decide which tools to use for automation testing?
It’s quite a broad open-ended question. And while that is good, there are just so many answers to this question that seem obvious if you’ve got any experience in software development. In my opinion, it makes it too easy to give a standard, uninteresting answer. On the other hand, it gives the interviewee the chance to ask about the choices of test automation and what we are using in our day to day work.
So maybe if you’re an interviewee and the interviewers haven’t asked you this question, or they haven’t really gone into much details about what automation testing tools they are using in their work, then this question would be a nice question to ask.
What types of tests would you not automate?
Yeahhhhhh. I like it. Interesting and encourages discussion. Allows for why questions and to see their thinking. And adding a not to get the interviewee thinking about the negative scenarios. And could even go into why only those tests? and maybe get into probing their unit testing knowledge (which should be standard knowledge for an experienced Automation Tester). There are also some pretty obvious answers that are not so correct and there’s plenty of room for discussion.
“Monotonous, repeatable tests shouldn’t be automated?”
Do you have any questions for me?
Classic. To be honest, it’s a bit of a staple question. I think everyone interviewing for a role expects this question to come. Like the standard why are you interested in this role?. I loathe this question, as it seems so boring and unimportant. No one will reply “because I want to get a job so that I can apply for a mortgage”. Instead, they will reply “I like to learn/want a new challenge/I’m interested in what company X is doing”. Any answer is fine for me. It’s just a good ice breaker and provides little to no value in how good they will be in the role.
This question makes it easy to read between the lines of an interviewees answer. They may have spent 8 months at each of their previous 5 roles and then say “I like a new challenge”. So it’s easy to think they‘re not interested in the role they are interviewing for and will jump ship as soon as there is another “challenge” they are interested in. It could be that those last 5 roles wasn’t a good fit for them, and they realised after 8 months. Maybe the work they were supposed to be doing never actually happened and they weren’t feeling good doing what they were doing.
A good interview question is one that does a few things:
- Gives the interviewee chances to display their knowledge.
For example: if you’ve been doing something in a professional capacity for any length of time then you’d be happy to answer related questions about your profession during an interview situation.
- Open-ended enough so that there can be a chance for the interviewee to possibly add other extra information.
For example: during your time at X you developed an Automation Test Framework. Tell me about it?
- The best interviews are ones that feel more like a conversation about the job.
- Gives the interviewee lots of options to ask about things that they will be doing in the role.
For Senior Automated QA Engineers it’s good to ask situational questions. An example would be how they would handle disagreements about solution design. Or maybe give them a controversial subject like CICD slows us down and is a burden to our productivity. Do you agree? I like this question, as it encourages discussion and a structured argument. The interviewee gets a chance to disagree (or not). And their answers show you a lot about how they think and if they have enough confidence to state their opinion.
There are a lot of interview question suggestions that are much better than the suggestions in the original article. In fact, there are so many questions that it’s a huge task to prepare. The best thing to do is to have a great understanding of the work that you do for a job.