This is the 12th episode of HackerRank Radio, a podcast for engineering leaders interested in solving developers’ toughest problems today: Hiring the right developers. You can subscribe to us on iTunes and Google Play.
Bazaarvoice is helping companies build their online reputation. Being the largest provider of ratings and reviews means they see upwards of 1 billion users’ devices on a monthly basis. Working for a company with this volume of data is exciting to a developer. We wanted to understand how Bazaarvoice approaches their tech recruiting strategy.
In this interview with Bazaarvoice Senior Director of Software Engineering Paul Hill, our SVP of Customer Success Gaurav Verma uncovers technical recruiting insights from the other side of the house: the engineering side. The two go into detail about building tech talent brands, building your candidate pipeline, and how to effectively engage with a candidate before asking them to complete a coding challenge. Take a listen to the podcast or skim the transcript below.
Jump ahead:
- Establishing a tech talent brand in a new geography
- “Candidates are not resources”
- Building your candidate pipeline---it's never too early
- Strengthening the recruiter/hiring manager partnership
- Engaging with candidates before asking candidates to complete a 1-hour coding challenge
Transcript:
Gaurav: Welcome to HackerRank radio, episode number 12 episode. I’m Gaurav Verma, SVP of Customer Success here at HackerRank. This week, we’re bringing you a conversation I had with Paul Hill, Senior Director of Software Engineering at Bazaarvoice. At our flagship event HR.main(), Paul and I dove into some of the most challenging aspects of technical recruiting today. From building your brand in a new city, to building your candidate pipeline---Paul shares some great insight from the perspective we don’t always get to hear from: the engineering side.
Paul: Paul Hill currently with Bazaarvoice in site lead in Belfast. My undergrad is computer science, so that’s been in computer for actually 14, I think, is when I first got introduced to them. As a site lead, I’m kind of just responsible for all aspects – recruiting, offers, culture. It’s extremely diverse responsibilities.
Establishing a tech talent brand in a new geography
Gaurav: Paul you have a different set of challenges that you’re the one who created the first team in Belfast, right? You started building up the team, the first team was built out in Austin which is headquarters for Bazaarvoice and then you took on this endeavor of saying, “I want to build a team from scratch.” Probably very different experiences, Austin Texas versus Belfast. Tell us more about that experience and the differences in building teams in two completely different geos.
Paul: Sure. So, the one thing about Austin when I joined 5½ years ago is that they already had an established brand, so it's not like we had to really work at building a brand in Austin. So, I mean, in Belfast there was nothing, no brand. I don't know how much people know about Belfast, but it’s a very clear close-knit group and the technical area is even more. It is not uncommon to run into three or four people just walking down the street that are in the exact same field as you. So, it is was very important to start off right and avoid going wrong when it came to building the brand, and to me personally being involved having the computer science background, talking about the technology, having been at Bazaarvoice 3 years at that time, I had a bit good experience across the different parts of the organization. I don't think I ever talked about recruiting at any of the meetups. Initially, that was not intentional, I just I forgot because I get excited about tech and I start talking about tech and I forget about, “Oh yeah, by the way, I'm recruiting too.” And the first seven folks we hired was very—overall, is going be a salesperson, and that's really what I was doing. I was selling the brand, the culture around technical all the things that have been said so far. What are the technical challenges? I didn't give a crap about the languages. I controlled my language. I didn't care about the languages that anyone knew. In fact, the first 5 folks— there were two that the only Java background they had was maybe a class in university maybe. So, it was ignore that and just figure out can I solve problems, are they willing to learn and that flexibility on my part enabled me to attract people that wouldn't have normally even considered applying.
One of the first groups that we got connected with, Dev Ops was the first but then the second group was the Women Who Code and I'm very proud of the fact we have twenty 23% of our staff as women.
Gaurav: That’s awesome.
Paul: I had to approach things differently for them. A lot of them wouldn't even apply because, “Well, it's really hard. I don't know if I can make it.” And so, I do a lot of convincing just to get them to take the HackerRank exercise and in some cases I couldn't even do that, so I had to get them in to actually write code on a whiteboard because they were really scared of it but super successful with it. That has enabled me to build a great brand in Belfast – well respected, at least that's what I hear. And now, we actually had a young lady, a developer, who got the in-person interview with us and quit her job the same day that we told her she had an in-person interview. She hadn't been even interviewed with us yet but she was that good, that confident and wanted to work for us that much that she chose to do that.
Gaurav: That’s excellent. We need a lot more of those, right?
Paul: Yes.
"Candidates are not resources"
Gaurav: Absolutely. That's great. When you and I talked and you've mentioned this before, which was very profound because we had HackerRank and it starts from—Vivek has a very similar, like, we have the same philosophy where you said candidates are not resources and this is something that we all have sort of, and it starts with Vivek, and he instills this as people are not resources. In the context that you talked about it was candidates are not resources and it's really important that the candidate in the company, sort of, that the alignment and that relationship is really important, and so, candidate experience has always been top of mind for you.
Paul: Yes.
Gaurav: Tell us more about that philosophy and how that's worked and what are the things that you've done as you scale up this team around the candidate experience.
Paul: So, Belfast had to adapt some and we've had to adapt some. So, if you go to Austin, the typical software engineer interview starts around 9:00 and finishes around 4:00, so it's a solid workday. I think there are 7 interviews in that and in Belfast the typical interview is a lunch hour interview spread out, you'll do one today and then maybe one in two weeks and then another one a couple weeks after that. So, it'll take a month or so or even two months before you finally get to a point where “Okay, we're ready to make an offer.” Well, that's too slow for my perspective. So, I had questions initially with the first group of folks we heard like, “Hey, can you change that?” And I'm like, “No, we're going to do the same thing because we need to see you're in assess. When things get broke, you've got to fix them and it doesn't matter what time of day or night or whatever.” So, we've got to be able to assess someone under pressure. And so, we made those kinds of shifts and we went from 7 hours to 4 hours. But by using Hackerrank, that four hours, it's still technical focus, but we also need to focus on culture. There was a comment, I remember, who said like “skill is the most important thing.” I don't agree with that it is very important. But if they don't fit, it doesn't matter how good the skills are from a cultural perspective. So, we've had to adjust some little things like that because Austin is a very close-knit technical community. It was very similar in Belfast, very close-knit technical community. So, taking the similar approach of understanding that if we do something wrong, take the wrong approach to diversity or culture or treating people like things instead of people, it will spread and we will fail. Recognizing that they're in control in reality. They can go get a job— if they're good then get a job anywhere and they're in control. I've got to realize, I'm giving up control here. All I'm trying to do is are the fit with us, are we a fit with them and then come to an agreement on how much to pay them.
Gaurav: Yes. I struggle with that one, even in the valley trying to make customer success, hiring for customer success, I'm not in control. They've got plenty of options to go to and that tough giving up control. Hung asked Lindsay and Bradley two questions and I'm going to want your perspective from the other side. The first one being your tech brand and both of them shared a lot of insight on developing the tech brand, how important it is, and what approaches they're taking. I would love to get your perspective it's a massive shift and what's working, what's not working.
Building your candidate pipeline--it’s never too early
Paul: One of the things I learned in Belfast is it's never too early and it struck me as odd in that there's a lot of focus on what's and a primary school and getting involved. PWC has a program there that’s kind of Dragon's Den approach where you get the kids pitching in technology projects and things like that. It struck me in there, my daughter is, what’s that, in Gen Z and she just finished her master's degree, but it struck me that I remember when she was 12 and she was interested in what I was doing and what I did and the company I worked for and things like that, and it's like, okay, it isn't too early at 12 years old to get them interested in technology and get involved as a company, representing the company and in events like this with the kids because in five years, they're 17 and they're either going to go to university or they're looking for an apprenticeship or they've already been coding for five years, right? And they're probably ready to go in terms of they could come in and start making an impact at the job in terms of developing the brand. So, from scratch, it was getting out there talking about it. It's basically everything you said getting out there talking about the technology and the challenges.
The one thing is, a lot of folks don't know who Bazaarvoice is or what we do. We are the largest provider of ratings and reviews in the world. No one’s close. I mean, we've got over a billion reviews in our system. One billion so big B. I think is like 1.1B right now. We see over a billion users’ devices on a monthly basis. That gives me goosebumps, but it also gives the developers goosebumps when I talk about that too. That's all I have to do to sell it. Let’s just talk about the technical challenges, the scale that we have to work with. The fact that no one knows us is like, well that's kind of cool that no one really knows us because you're operating at scale like Google, Amazon and Netflix and having the same problems and challenges and when faced with that, is like, “I really want to work on that.” So being flexible is really been important especially with moms. I have had two candidates, one man and one woman who applied because we put part-time positions out there. But when we talked to them, the gentleman wanted to work on an open source project one day a week like, okay, you don't need to work part-time for that. Take your day and go work on your open source project. And the lady had two kids that needed to pick up her kids twice a week and leave the office at 2:30. Her former previous company, you have to be part-time to do that. Why? It’s nuts. So, they’re both full-time employees and but that reputation spreads, and just that part-time listings are spread widely within the women in the code community and we got applicants because of that.
Gaurav: There are so many elements to your tech brand. It's not just what languages, what problems you're solving but this is a multifaceted thing that we need to approach.
Strengthening the recruiter/hiring manager partnership
Let's move to the last question and I want to give some time to everyone in the audience to ask you a question as well as folks in the livestream. Recruiting and engineering, while we are partners, we think differently. How do you work together? It’s like, do you have a C level executive who's driving this top down versus it's grassroots efforts? From your perspective when you're on that table with your recruiting team, are you seeing alignment or are you seeing dissonance and how are you handling that? And what's your advice to everyone in the room is, how should they be approaching?
Paul: My philosophy on software engineering is quality first. I think it's like that in the long term. That's the least expensive way of approaching it. So, my philosophy in recruiting is quality first. I don't care about the volume. I care about the end result and having the best quality software engineers. I would say my recruiter and I are almost always on the same page with that. It's actually us arguing with the, “Hey, you're not going fast enough, you're not hiring in the field.” I think my target was 55 engineers by the end of our fiscal year, which was on May and I had, including the placement students, which are absolutely outstanding, I had 49. It's like success to me, but from the C level it’s like, “Well, we need to look at another location because you're not hiring fast enough.” I’m like, “Okay. I mean, what are we talking hiring fast enough? A hundred a month? Five a month?” But when it comes to quality, let’s say, by far the best engineering team I've ever worked with and I don't I don't like fight battles for them, my product team does because these like these are the best engineers years I've ever worked with. They're the most collaborative engineering team I've ever worked with, like, “Okay, I don't have to say anything.” Long-winded way of saying we're on the same page, the way we did it was like when a recruiter join, we just sat down every single day, went through reqs, went through candidates went through job descriptions, they're learning –she was learning about the company, she’s learning about the technology and how to recruit the talent for Bazaarvoice and now I don't do much with her because she handles a lot. Not all of it, but closely with the different hiring managers to handle the hiring. I think they've got a great working relationship because she's been on the same page with us from early on.
Gaurav: Definitely. That’s great. I want to make sure that we have enough time for questions from the audience and then folks on the livestream, so please I'm sure you want to know a lot more about from the engineering side of the house or your business partners. What would you like to hear more from them?
Engaging with candidates before asking to complete a coding challenge
Participant: Just wondering about asking someone to do a one-hour test prior to a sort of a strong application engagement in the process. How has the feedback been on that?
Paul: So, we have three views on candidates. So, we have applicants sourced and referrals. We never on sourced and referrals give them a test first. It's always engagement with at least a recruiter or a hiring manager first and then as part of that, we describe whoever was describing the product and tell them what the process was. We tell them, “Here's the process that we go through which includes this technical screen using Hackerrank.” With applicants, it depends on how many, so like our placement applications those tend to be in the hundred or so range, so we don't talk to them first. We basically send them the mass email with the Hackerrank exercise and those that complete it look at the results and those that don't, then we give them I think 14 days or something like that or one renewal of the link and then we send them a— well, we don't even send him a note in that case, they basically disengage. So, we looked at it as the different types, so we don't just automatically give them the exercise first.
Gaurav: We find that a lot of our customers when it's volume or someone's applying even having that sort of apply here and there's a go take the test. And even in the instructions for the test, give them more visibility on once you take the test what can you expect the next step to be, like, how quickly before you respond to them and they will take it. Especially when you're applying on your careers page, you just have like an “apply here” button and take the test and you will see the higher sort of acceptance rates for wanting to spend the hour.
The other piece that's really important is based on the role that you're hiring for is the content. Are the questions that you're asking, are they and are they contextually relevant? So, someone that's a little bit more senior, you know, 5+ years of work experience or even 2-5 years of work experience, if we design assessments which are more textbook, straight like college, I have to go remember this is what I did. It’s algorithm that I don't remember anymore, I don't use it every day. I have to go crack open a book, then there's a sort of a “do I need to go put in the effort to do this” versus “let me give you something that's more real world.” And so, one of the examples that whatever we are showing you is bring something from your own core base. It's an opportunity to solve to the candidates as well even if they're just applying or they’re getting sourced and saying, ”Here's the type of probably we're going to be solving, go solve this and experience it.” And then that we will see a much higher acceptance rates from them. So, there's lots of different approaches and techniques and depending on the source how they're coming to you, the experience level is sort of different workflows that work really well.
Paul: And we tend to try to pick questions that are related problems that we saw versus creating our own problem. It’s like, we have this problem let's find something that's somewhat similar.
Gaurav: For sure. Yes, there's a big library for you to start from.
Great. Anyone else? Yes sir.
Participant: You have mentioned about the mom returners and how you almost had to persuade them to take tests and kind of engage in the process. Just curious as to how you started those conversations with them in the first place, where you were finding them and how you could have worked on that diversity inclusion, please?.
Paul: So, it's not wasn't just moms, it was just basically women regardless and there are almost always referrals. I can't think of any that were— the one that quit her job first then did the interview, she was a referral but she obviously was very outgoing in terms of taking chances and taking risks. And then the other ones, so we did very early on July 13th of 2017, we did a Women in Tech or Women Who Code talk on kanban and culture in that. Out of that, we got one person that was interested and then we hired her then she kept connecting us through that community. And so, that's basically has created this series of referrals. I think there is only one of the women that was not a referral. I think all the other ones were a referral of some sort.
Participant: Thank you. Yes, I was wondering if you kind of did any personalized outreach, I don’t know, if help your recruiter with—
Paul: Basically getting involved in everything women and tech community whether it's Women in Tech Women Who Code, Women Techmakers. I sponsor them. Because I do have a lot of folks that already participate there, they give me a heads up when events are coming up and I just sponsor and it doesn't cost a lot of money. I might meet up maybe £250 or something like that for like pizza and drinks and things like that. The main thing I really push for and this was Women Techmakers, they wanted us to be the platinum sponsors, like, I'll be the platinum sponsor as long as I get a keynote because getting somebody up there talking about our tech and our culture and the key to expanding the brand. If I can at least get them one-on-one or two-one-one where I can get them talking about their experience, I can maybe get them over the hurdle of taking that technical exercise At least one of them was so nervous that I basically did a whiteboard with her and it's like, I think you should just interview with us and she did and we made her an offer and she joined us a few months later.
Gaurav: It's awesome. Thank you so much for your time and your insight. It's been awesome. Thank you.