Conduct the interviews

Before we talk about what questions to ask (and not ask), you need to make your interview questions the same for all candidates. And be sure that each interviewer uses the same questions with each candidate.

Why?

Because you’re comparing each candidate to the same job description, the opportunity is ripe to inject some useful structure into the evaluation and decrease the chance of a mis-hire. You are also setting some boundaries to reduce the risk that an interviewer will ask illegal or unethical questions (see subsection below).

To further reduce this risk, and to better screen candidates, be sure to limit the questions to the skills, knowledge, and abilities that are required to be successful in that job. This should be straightforward to do because you already know what kind of behaviors and skills you want from your ideal hire for this position.

An unexpected benefit of this approach is you are helping each interviewer gain a depth of understanding about what this role is really about and what they can expect from the person that you ultimately hire.

Another benefit of this approach is candidates will see the interviewers as all being on the same page in terms of what they expect from the person who lands the job. This will reduce confusion all around and better prepare the successful candidate to hit the ground running from day one.

When to involve your current team members

Bringing a new person onto your team is a big deal. You want to pick the right person and you want that person to succeed. While a new person’s ability to succeed is a function of their own abilities, it’s also a function of the daily support they receive from their new team members. Unless you have a strong reason to distrust your team’s perspective, involve them in the interview process and listen to their recommendations.

Of course, the hiring manager needs to oversee all the interviews conducted by others. And, they often conduct an interview with the final candidate(s) before deciding who’s going to get the offer.

In contrast, you have a wide range of options for involving your team members. At one end of the spectrum, you could involve some or all of them heavily throughout the interview process, which we believe is ideal. Or, their involvement could be nothing more than welcoming their new co-worker after you make the hiring decision without their help, which is usually not a good idea.

We find there are two major variables driving team involvement:

1. Available time
2. Role they will play

Available Time

Many hiring managers feel crushed by a never-ending tidal wave of urgent tasks. They struggle to find time for tasks that are important, but not urgent. Having current team members help select their new co-workers is a great example.

It may seem better to keep your team members focused on urgent matters to avoid missing commitments to your internal customers.

But that might be a big mistake. Bringing a new person onto the team will be disruptive to existing team members if they don’t already know them. Giving the team leaders time to help select new team members can greatly reduce that disruption.

Where will the time come from? We believe it’s a matter of prioritization, which you need to do all the time with your budget, so you have the ability to make similar priorities with team time. You might have to renegotiate a deadline to make it happen, but it will pay back many benefits in terms of how fast your new hire will be able to come up to speed with the help of their new team.

Role they will play

If you commit to having your team members involved in the interviewing process, it will help a lot if you are clear about what role they will play.

Will they each conduct a 1:1 interview with every candidate? Is their role to simply make a recommendation about who to hire, or do they get a formal “vote”?

However you want to involve them, and you should make their role clear.

How To Get HR To Conduct Initial Phone Screens Effectively

• Goal for HR involvement: Find and remove candidates early in the hiring process who display a big reason why they are not a good fit for the organization and for your team based on the job as you defined it and based on the criteria you set.
• Not a goal for HR involvement: To conduct the one and only interview to determine “fit” for your team.
• How to achieve the goal: Build a strong working relationship with your HR rep so they know what you’re really looking for in a candidate.
• What if you can’t achieve this goal: Minimize HR involvement, or conduct a second “initial” screening call with everyone that HR screened. If you end up in this situation, despite your best efforts, be sure to not burn your bridges with HR. While our Ideal is to “Hire in a way that builds relationships with candidates, even when the decision is not to hire someone” it will help you in the long run if you build the best relationship you can with your HR rep. Who knows? Things change and if you maintain contact, you may have the opportunity to build the kind of partnership with HR that you really want.

Handling Candidate Personal Information

During the process of reviewing and interviewing job candidates, we are exposed to a lot of personal identifiable information, or PII. Protection of this information is a responsibility of everyone involved in the hiring and employment process. While there are countless laws that relate to the handling of this information, there is a common thread among them all. That is, maintaining information only for as long as required for the purpose in which it was collected and used. After which, the data should be purged from systems.

This is especially important considering we may receive hundreds of applications for a single job posting, making us a greater target for malicious actors who may want to exploit job seekers or commit identity theft.

While we are considering candidates, there may be information shared or provided which we should be cautious in our handling of, through the consideration process:

• Contact Information. The contact information of a candidate should be limited only to those who need to contact the candidate. Otherwise, it should be marked out when the resume or CV information is shared.
• Health Information. A potential candidate may share information related to a health condition in which accommodations may be required. Should this occur, the information should only be shared with HR for consideration of accommodations and otherwise not documented unless otherwise determined by HR or Legal.
• Identity Verification. A hiring manager may be responsible for the collection of identification documents that are required to confirm the candidate is legally allowed to work. This information should preferably be provided by hand, but should you need to receive them digitally, it should be only via secure methods. Sending the candidate a URL to securely upload their identity verification directly to your server is generally a good approach. In contrast, email is not a secure method on its own, so if a candidate emails their documents, make an effort to remove it from your email database so it won’t get caught up in backups, archival, or email journaling systems.

As always, it’s best to verify any specific requirements and processes with your Legal and HR teams as they may include reporting or documentation obligations not otherwise discussed in this section. Further, having a documented process for how you are handling this information will also benefit the organization, should one not yet be established.

If you are not familiar with your municipality’s laws and you operate in the United States, the National Conference of State Legislatures has a great succinct breakdown on each State’s laws which can also help you in your understanding of laws specific to your area: https://www.ncsl.org/research/telecommunications-and-information-technology/data-security-laws.aspx

Ask Better Questions to get Better Answers

If you’re struggling to find great talent, I can almost guarantee you that your current interview questions are holding you back.

Remember, hiring for talent means that we need to get to know each candidate as deeply as possible as individuals.

• Who is this person?
• What makes them unique?
• What was their childhood like?
• What difficulties have they overcome?
• How do they make decisions?

These are the things you want to find out about a person during the hiring process. Because great questions will lead to great answers. And great answers will lead to great hires.

But what does a “great question” look like? I like to think of it this way: a great question is like paint. You, the artist, and you use great questions to paint a deeply rich picture about a person. You start with a blank canvas, and little by little, question by question, you paint a picture of this person. The level of detail and depth of this picture depends on you and the quality of your questions.

So what does this mean in practice? Don’t ask simple questions like “tell me about yourself” and then stop after they answer. Dive deeper into their answer.

• Ask how they got here.
• Ask where they came from.
• “Tell me about yourself”, but more.
• Get to know this person and imagine being in their shoes as they tell their story.
• Ask questions that PUT YOU THERE in their story, like you lived that experience with them.

You need to understand a person’s past behavior in order to understand their future behavior. In other words, use historical data. Ask what this person has accomplished in their past and then consider if that translates into success in the world of cybersecurity.

Still stuck? There are 2 categories of questions you want to ask.

The first is “Ability to learn questions” - questions that uncover someone’s ability to learn something new or difficult such as What new thing have you learned recently? How did you learn that thing?

The second category of questions uncovers someone’s “Ambition” - questions that reveal someone’s curiosity and drive. When you ask these questions, take a look at their demeanor. If their eyes light up when they answer, then you know you’re on the mark. Ask them, what topics are they intensely curious about? Or what do they do to satisfy their curiosity?

Remember, your goal is to ask questions in order to know someone at a deeper level. Once you get to know them better, not only will you understand their potential, but you’ll know if they’ll be a good fit for your team and your organization.

Here’s a short list of example questions to help you get started:

Example Ability Questions

These questions uncover someone’s ability to learn something new or difficult.

TIP: Remember, cybersecurity can be learned! Your goal with these questions is to understand a candidate’s ability to learn. If a candidate has learned other complex topics, then will be able to learn whatever they may not know yet about cybersecurity.

“What new thing have you learned recently?”

Follow Up Questions:

• How did you learn that thing?
• What was your learning process?
• Did you follow a structured learning plan? Or did you just go with the flow?
• Did you have help? What resources did you use?
• How long did it take you to learn that skill?
• What would you have done differently, if you could learn it all over again?

Things to note as an interviewer:

• The thing doesn’t have to be technical, but it should be difficult. Did they share something they learned which required significant time, effort, and critical thinking to learn it?
• In addition to time, effort, and critical thinking, look for evidence of resiliency. Did they try to learn this thing in the past, but gave up? Why didn’t they give up this time?

“What difficult problem did you solve recently?”

Follow Up Questions:

• How did you come up with the solution to solve the problem?
• What resources did you use to solve the problem?
• Did your solution have any gaps or holes? What did you do to close those gaps?

Things to note as an interviewer:

• The “problem” can be from any part of the candidate’s life - personal, academic, or work-related. You may want to reassure your candidate that they don’t need to share specific names or other personal details and focus on sharing the process they used to solve the problem.
• Various problems will require different types of solutions. Note if the candidate leveraged a variety of sources in order to find a solution to their problem.

“What have you failed at recently?”

Follow Up Questions:

• What was the root cause of the failure?
• What did you learn from that failure?
• What changes in your life or career have you made as a result of that failure?
• What are you doing to make sure that you don’t fail at the same thing again in the future?

Things to note as an interviewer:

• Remember that failure is not necessarily a bad thing, as long as growth resulted from it.
• Use this question as an opportunity to assess the candidate’s level of self-awareness:
• How do they describe themselves and others in the story?
• Do they blame others? Do they blame themselves? Or do they take responsibility for their part in the problem?

“If you were asked to learn about [name of a technology or tool that they would use on the job, but don’t already know], what would be your first step?”

Follow Up Questions:

• Why would you start there and not by [other path to solution]?
• What other solutions might you leverage?
• What would your learning process be?
• How would you know if you were successful in learning that new technology or tool?
• What would you do with the new knowledge that you just gained?

Things to note as an interviewer:

• Use this question to learn about a candidate’s learning process. Understand the details of how they structure their learning, and what types of learning goals they set for themselves.
• Everyone learns differently, so there is no right or wrong answer, just different answers. If a certain learning process works best for someone, then that should be celebrated, not criticized.

Example Ambition Questions

These questions uncover someone’s ability to learn something new or difficult. To uncover someone’s curiosity and drive.

TIP: When you ask these questions, take a look at their demeanor. If there is a shift when they answer, then you know you’re on the mark.

“What personal or career goals have you set for yourself?”

Follow Up Questions:

• What steps are you taking to achieve those goals?
• What do you like about this position?
• How does this position align with your goals?
• Do you think your goals are challenging or easily achievable? Why do you think that?
• What’s the biggest investment you’ve made in yourself to help you achieve your goals?

Things to note as an interviewer:

• Look for clarity on the candidate’s self-awareness and understanding of where they’re starting from, as well as clarity on their destination.
• Challenge the candidate to think critically about WHY they chose their career goals, and how they might overcome specific pitfalls and challenges.
• Is there a topic or area in security that are you intensely curious about?

Follow Up Questions:

• Why are you curious about [topic or area]?
• What have you done in the past to satisfy your curiosity?
• What would make you stop being curious about those topics?

Things to note as an interviewer:

• This question allows the candidate to talk about something they are passionate about and might have already explored on their own.
• Challenge assumptions on why they are intensely curious about that security topic or area, such as “Hackers in movies look cool so I’ve always wanted to do pen testing”.
• Like the ability to learn questions, this question should uncover a bit about the candidate’s learning style and capacity to learn new concepts

Reflection: How good are your interview questions?

Before we move on, I want you to take some time to reflect on your interview questions. Think about the questions you asked at your most recent interview. Write them down if it helps.

Next, use this grading rubric and compare each question with the scoring table and give it a grade from 0 to 4, 4 being the best. The rubric has a detailed explanation of each possible score. Be as objective as you can with the scoring.

Once you’re done rating all of your questions, Tally up the score, then average those scores.

If your average is 3 or higher, you’re doing a wonderful job. Ask someone else in your organization to grade you as well, and see if they come out with the same scores.

If your average is between 2 and 3, you’ve got room to improve, but are on the right side of halfway. Are your questions consistently good, consistently bad, or a mix of good and bad?

Finally, if your average is between 1 and 2 or even below 1, you have your work cut out for you. Consult with your HR team or a trusted mentor to improve your interview questions. You can use this grading rubric as a guide. Aim for all 4s as you write your new questions.

It’s okay if you didn’t end up with the best scores. If you have room for improvement, use the grading rubric to help you create better interview questions.

Remember, there is value in the journey. Have a growth mindset and push yourself through the challenge. You’ll be stronger for it.

What Not to Ask During Interviews

Hiring employees is a highly regulated activity. There are laws and regulations at all levels of government designed to prevent illegal discrimination “against a job applicant or an employee because of the person’s race, color, religion, sex (including pregnancy, transgender status, and sexual orientation), national origin, age (40 or older), disability or genetic information” U.S. Equal Employment Opportunity Commission (EEOC).

You can expect to see restrictions at the federal, state, county, and city/town levels. The effective Cybersecurity Hiring Manager knows and follows these rules for the jurisdictions where their business is operating.

During the interviews, you need to stay focused on relevant questions. Even if the candidate offers information that you’re not allowed to request, that’s not an open door to ask illegal or unethical follow-up questions.

Questions that are not allowed during the interviews are typically those that have nothing to do with the candidate’s ability to do the job. More to the point, these questions are usually related to illegal forms of discrimination.

When interviewing candidates it’s important to focus on questions topically relevant to the position, their ability to perform the job, and what they can contribute to the team. Some questions can end up being illegal, and others are just generally not useful in determining the viability of a candidate. While our curiosity and interest in building connections with people can lead us down the point of asking different questions, we can find ourselves asking information that can open us up to legal action or an EEO claim.

When in doubt, don’t ask. While the below are general guidelines, we are not your legal counsel and we recommend any questions you feel are necessary that could be in the below mentioned areas be evaluated by a legal professional.

Because laws vary from location to location, we can’t give a full and accurate list of all the questions that are prohibited. You can get the list that applies to you from your human resources partner.

So, what questions should we avoid in an interview?

• Age related questions. These can be direct such as asking how old you are, or can be surmised based on when someone graduated from High School. This also applies to questions around retirement, comfort around certain ages of employees, and so on. If the work being conducted requires people to be above 18 or 21 for liability reasons, you can ask if they are over that age, and nothing more.
• Prior salary. A candidate’s prior salary information is not important to your hiring efforts and questions on this are becoming increasingly against the law. If you want to avoid spending time with candidates whose salary expectations may not align with your budget, post the range in your job posting or mention early on that the job has a pay range, cite the range, and ask them if that is something they can work with.
• Individual traits. Avoid any questions related to race, gender orientation, gender representation, sexual orientation, traits, visual differences, height, weight, or genetic-related information.
• Family or living situation. A candidate’s family or living situation, while tempting to ask, is a prime area where our assumptions or biases can get the best of us. Asking questions around spouses, kids, pregnancy or living situation are off limits. Even if a candidate voluntarily provides this information, you should avoid documenting them to avoid influence on your decisions. If the employee is remote, limit questions to things related to the job such as “is there an area of your home you can work without distractions?”.
• Use of public transit. How an employee gets to our workplace should not matter and can fall into discrimination around financial status. Instead, we should be focused on if they can arrive by a particular time. The rest of the details are for the candidate to address by themselves. If the job posting requires use of a personal car for conducting company business, you can ask if they have one. Though, recognize that said personal vehicle requirement is likely disqualifying other viable candidates if they haven’t had the opportunity to acquire a personal car.
• Arrests and convictions. Questions about convictions must be limited to those that are related to the job, if the job is sensitive, or requires government security clearance. Questions around arrest are more limited in nature depending on your municipality, especially as an arrest is based on suspicion and not a legal judgement. This resource from Nolo.com can be helpful in additional to advice from legal counsel: https://www.nolo.com/legal-encyclopedia/state-laws-use-arrests-convictions-employment.html
• Nationality or ethnicity. Any questions, even related to the language or dialect a person speaks, are a means to deduce someone’s national origin or ethnic background and grounds for an EEO complaint. The exception here is if a particular language is necessary to be spoken as part of the job that is not the language you are actively speaking in. Beyond that, you can ask whether they are authorized to work in the country.
• Religious background. These questions are often related to the availability of a person around the holidays. Avoid references to specific religious holidays and focus on their availability as it pertains to the role.
• Disability and health status. Disabilities and health issues can be both visible and invisible to us during the hiring process. Even if you never intentionally discriminated against someone with a disability, you can be set up for an accusation that you did if it’s something you asked. Your focus should instead be on whether the candidate can perform the job. Should they need accommodations, the candidate should bring that up after you’ve made the offer. While there are some exceptions to whether you can ask about accommodations, that is a question best left to your HR partner or legal counsel.
• Military Service. While asking about a candidate’s military service is generally allowed, avoid questions relating to the type of military service and reason for discharge unless it is explicitly required for security clearance. Instead, ask what training or experience they received that relates to the position.
• Prior Job Experiences. Asking a candidate about their prior job experiences primes the candidate to be negative. Clearly, there is a reason why they are looking for a new job, but all you need to focus on is whether they are a fit for your job opening. Instead, focus on questions as to what brought them to apply.
• Unemployment or Gaps in Employment. It’s OK to ask about gaps at a high level. However, asking for details about gaps of employment or unemployment should generally be of no concern in determining if a candidate is a good fit as there are any number of reasons for it. Such questions could wade toward discrimination claims if it related to family related matters, health related, and so forth.
• Finances. Asking candidates for information details about their personal finances is not appropriate and has no bearing on them conducting their job. However, it is reasonable to ask whether the candidate can live without excess stress on the compensation you offer.

There are also many kinds of unethical questions that you should avoid. In part because they aren’t relevant to the candidate’s ability to do the job. And because they can lead the interviewer into asking illegal follow-up questions. For example, stay out of these areas:

1. Whether they drink alcohol and how much
2. Any topics related to dating
3. Questions that are designed to indirectly uncover information you are not allowed to ask for directly, such as their year of graduation from high school or college

Here are some example questions that you do not want to ask candidates:

1. Where were you born?
2. Are you going to want to take time off for religious holidays?
3. Do you plan on getting pregnant/having children?
4. When did you graduate from high school?
5. Do you have a disability?
6. Have you ever filed a workers’ compensation claim?
7. How did you learn Spanish?
8. Are you a U.S. citizen?
9. How much longer do you plan on working?
10. Where’s your accent from?
11. Do you have children?
12. Do you need Sunday mornings off for church?
13. What’s your background?
14. Have you been really sick in the last year?

As you can tell, some of the above questions could be asked without your intent to discriminate illegally. But candidates can’t be expected to figure that out by themselves. To be on solid ground, we suggest writing up your questions in advance and then asking your human resources partner to review your list before you start interviews.

Overall, your focus in evaluating a candidate should be on their ability to perform the job based on the description you provided, whether they satisfy the requirements you have set forth in the job posting, and if they’ll be a good fit. You can worry about getting to know them more after an offer has been extended and accepted.

Qualities to Look For

1. Self-starter
2. Human (soft) skills: critical thinking, ability to influence & persuade, empathy and emotional intelligence, ability to quickly learn (and apply) new skills, tools, and concepts, humility, being approachable, reliability, resourcefulness, curiosity
3. Willing to teach others and document to enable teaching future employees

Effective Interviewing Techniques

Stefanie Hoffman: It’s important to remember that not all candidates, even ones that are very skilled, interview the same way. Techniques like asynchronous interviewing, outlining the interview “itinerary” or what the candidate can expect ahead of time and interviews in which the candidate can demonstrate their skills should all be considered. Also, employers should demonstrate that they are willing to make accommodations for neurodivergent or disabled candidates.

Ineffective interviewing techniques (avoid these traps)

[TBD]

Technical Interviews Best Practices

Technical interviews are extremely important for every role and level in cybersecurity. After all, this is a multi-faceted, highly-complex, and deeply technical discipline.

This may surprise you, but there are many people trying to break into cybersecurity that have a deep understanding of cybersecurity concepts. This may seem contradictory, but I assure you, it is not.

Why? Because a deep understanding of technical concepts can be learned outside of a work environment. Book knowledge always precedes the application of that knowledge. Frameworks, protocols, reference architectures, and best practices - entry-level folks learn from the same resources that you use yourself. They, just like you, put in the effort to understand and comprehend. They, just like you, are able to see the bigger picture.

Someone who is able to learn difficult concepts, and deeply - is someone that you want on your team. When doing a technical interview of entry-level candidates, ask them to explain a very complex topic to you as if you were a child. If they can do this well, chances are, they have a very deep understanding of that topic.

As the saying goes - “If you can’t explain it simply, you don’t understand it well enough.”

In other words, look for someone who can explain a very complex topic in very simple terms. Avoid “ trivia” interviews - like what port is SSH on, or what is hashing? These answers can be found online and memorized. They do NOT demonstrate if someone has an understanding of a concept or WHY it exists.

Instead, ask WHY.

• Ask WHY specific security controls are used in certain situations.
• Ask WHY a particular vulnerability exists, not WHAT the vulnerability is.
• Ask multi-layered and contextually-rich questions using real-life scenarios.

Ask your entry-level candidates WHY, not HOW or WHAT. Because no matter your level in cybersecurity, you must always know WHY security controls must be implemented, and WHY vulnerabilities exist.

Entry-level candidates haven’t yet had a chance to figure out HOW to implement the security controls, or HOW to mitigate vulnerabilities. And that’s okay.

For entry-level, it’s okay not to know HOW, but it is imperative to know WHY. Always ask WHY.

Take-home Projects Best Practices

If you’re looking for a better way to evaluate technical skill, use take-home projects.

A take-home project is a short assignment that you can use to evaluate candidates on how they would perform as an employee. Candidates have the freedom to use whatever resources they need, in a less stressful environment.

What does a good take-home project look like? Let’s take a look at this one:

Entry-Level Cybersecurity Analyst - Take Home Project

Read the following whitepaper, “SANS 2022 ATT&CK and D3FEND™ Report: Incorporating Frameworks into Your Analysis and Intelligence” and answer the following questions. Be detailed in your responses. You may use other sources and references. Note: You may need to create a free SANS account in order to download the whitepaper.

• Please submit this within [X] business days. This project should take no more than 90 minutes to complete.
• In your own words, what is the author’s main goal or objective in writing this whitepaper?
• In your opinion, what is the main benefit of incorporating the ATT&CK and D3FEND frameworks into an organization’s security operations?
• How might an attacker use the information found in the ATT&CK and D3FEND frameworks against an organization?
• Choose any attack technique from the MITRE ATT&CK framework. Explain this attack technique as you would to a child.

Take Home Project Next Steps

An entry-level Cybersecurity Analyst should have a basic ability to analyze and interpret data. So the take-home project should have one or two questions that directly assess these abilities. Take a look at the questions in our example to give you an idea of what to ask.

Once the projects are submitted, review them using a grading rubric. Here’s an example. Be sure to assess your candidates as objectively as possible.

Next, have two or three people from your technical team ask the candidate questions about what they did on their project. The candidate should explain WHY they answered a question a certain way and what they thought of the project itself.

The goal here is two-fold: firstly, to see how deep your candidate’s knowledge is on the concepts covered in the project, and secondly, to see how well your candidate can interact with members of your team. You’re essentially getting both a technical interview AND a behavioral interview in one!

Finally, remember to be respectful of candidates and their time. Not every candidate will have hours of free time to dedicate to a take-home project.

I create take-home projects that can be completed in two hours or less. And I make sure that I only have my top candidates do the take-home project. I never use take-home projects as part of the screening process.

Panel interview best practices

[TBD]

Effective Types of Interviews

1. Cultural fit (2-way street) Stefanie Hoffman: Interviewer should have some understanding of the candidate’s cultural background and how that will likely affect or influence their responses.
2. Hard skills (this should be defined and/or scoped to fundamental skills for that role, we can also emphasize that all technical skills can be learned)

Interview Questions

1. Behavioral
2. Technical
3. Cognitive
4. Personality
5. Workstyle
6. Human (Soft) Skills
7. Aptitude

Interview Answers

1. Using a scoring system
2. Red flags
   1. Misused security or technology terms
   2. Outright lying or gross exaggeration
   3. Manipulative responses
   4. Unable to articulate how they personally contributed to the success of a project (flag: using “we” instead of “I” a lot) Stefanie Hoffman: Is this a red flag? Some candidates, and often women, will naturally give credit to “the team” having been conditioned not to “brag” or openly market themselves, even if they personally contributed a lot. There are way to ascertain a candidate’s personal contribution without just paying attention to the pronouns they use.
3. Green flags
   1. Good understanding of the nature of risk
   2. Thinking and consideration beyond technology problems
   3. Self-driven to research or learn
   4. Active contributor to the community
4. How to spot potential on-the-job integrity problems
5. How to spot exaggerated skills or experience
6. Panel interviews
   1. Who should be involved
   2. What questions each panel member will ask
   3. How to ensure candidate feel safe to answer panel questions