My Experience at Insight Data Science

By: Ari Silburt
Date: Nov. 1st, 2018

Insight Data Science is a 7 week program designed to transition individuals from academia into careers as Data Scientists. From June - August 2018 I attended Insight in New York City as a Data Science Fellow, and for my project I built git-screened, a web-app that automates GitHub repository assessment (see here for more details about my project). Through Insight I got a job as a Software Engineer at Bloomberg, and am very happy about it. This blog post is intended to share my experience at Insight, along with the pros and cons of the program and how incoming Fellows can best prepare.

The TL;DR Summary

Insight was worth it. I ended up with my dream job at Bloomberg and it's unlikely that I would have gotten the job without Insight. Not because I was previously unqualified for the role, but because Insight has connections to high profile companies and thus the ability to put your resume at the top of the pile. Insight genuinely cares about you, and will often go above and beyond their contractual obligations. For example, one program director fought on my behalf to move things along quickly at Bloomberg. Since I already had a job offer at another company (with a firm deadline approaching), Insight was guaranteed to get their commission. However, since Bloomberg was my top choice, Insight went out of their way and expended additional energy to help facilitate a fast interview timeline.

This being said, Insight was not without a ton of stress, including a moment or two of crying at 2am (I'm not usually a crier). Not to mention, you will likely need at least 10,000 USD for living expenses. If you don't need/want (new) employment on a fast timescale, my recommendation is that Insight should not be your first option. Instead, I would first apply to the jobs you're most interested in (while keeping your old one) to see whether you can get a job without the stress/financial burden of Insight. If unsuccessful, only then would I apply to Insight.

The Cons of Insight

I'll start with the cons of Insight, but first, a bit of background on the structure of the program. The first 3 weeks are spent building a project, from which you make a 5 minute demo to present to companies and (hopefully) get an interview. If you don't finish your project, you can't make a demo to present at companies, won't get an interview, and won't get a job. Each phase in the program depends on successful completion of the previous phase.

For me, the project phase was the most stressful, especially since I was late figuring out a project. Most fellows land a project idea by the end of week 1, but I didn't until Wednesday of week 2. I thus felt 3-4 days behind everyone else (which is significant in Insight time), and early week 3 was having doubts that I would actually complete the program. During week 3, when I wasn't eating, sleeping or using the bathroom I was working on my Insight project, and intensity-wise I felt more stressed during this single week than I can remember (more than my thesis, etc.). I knew going into Insight that "it would be hard", but nothing could have prepared me for the real intensity of it. One analogy might be becoming a parent (I'm not a parent myself but have heard this from Fellows who are) - you can think, "okay, I'm not going to get a lot of sleep, I may not see friends as much, I know it will be hard", but it's a different story when you're actually in the trenches going through the experience and trying to maintain balance.

Since you're building a project in 3 weeks (and all your previous experiences have been in academia) most projects never reach their full potential and can have some real weaknesses. Fellows may end up presenting less-than-ideal versions of their project (and thus themselves) to companies, which can be tough to constantly do, over and over. In addition, as time moves forward and Data Science becomes more mature as a field, it seems to be harder and harder to develop a project at Insight that a) can be done in 3 weeks, b) provides genuine business value, and c) no Fellow/entity/company has done before. In addition, once you land that first interview at Company X the project you stressed over basically becomes irrelevant. I can't speak for all companies, but none I interviewed at asked about my project after the intial presentation. This all being said, I think it does generally look good on your resume to have completed a project in the fast-paced environment of Insight (which does provide long-term value beyond the demo phase of Insight).

Before coming to Insight I was a postdoc in Astrophysics at Penn State University, and had to formally leave my postdoc, move to NYC, attend Insight, and land a job before my savings dried up. Insight doesn't cost any money to attend, but the cost of living in NYC with no income is expensive if you're not prepared for it. Door to door (leaving Penn State -> first day of work at Bloomberg) I roughly spent $12,000 USD, which is not crazy, but is also definitely not nothing. This also excludes the cost of finding a permanent place to live, which is at least another $5,000 USD. There's also the added mental cost of living in an AirBnB out of two suitcases for ~3-4 months. I consider myself to be a pretty frugal person (e.g. I packed lunches most days), and was among the first wave of people getting jobs in my cohort. I should note however that part of the $12,000 USD cost came from:

  • flying back to Canada, staying with family to await my TN visa documents (I'm Canadian, eh), and flying back
  • attending a wedding over the summer

Funny enough, another member of the Astrophysics department at Penn State left academia roughly 2 months after I did and got a job in NYC at a company Insight partners with. However, they did not go through Insight, and passively applied to jobs (while keeping their old one) until they landed one. That likely resulted in a lot less stress, not to mention, constant employment/pay. I would probably recommend this option first to anyone trying to get a job in Data Science if you currently have a job and/or don't mind being patient. The worst that happens going this route is you don't get a new job and apply for Insight like you were going to do anyway. In addition, most (~75%?) people going through Insight land a job within a couple months of the program formally ending, but there's a long tail to that distribution where a few candidates can take 6-12 months to land a job. There were two candidates I met from previous sessions who had finally gotten jobs after 6 and 8 months of searching.

Lastly, although I was personally happy with the distribution of companies available, some Fellows in my cohort expressed displeasure. For some, only one or two companies out of 30 were remotely appealing, and if they didn't get those jobs they had to either a) interview at non-ideal choices or b) start looking elsewhere. The roles available in my session ranged from consulting (stats heavy, client facing) to ML engineering (machine learning/CS heavy, non-client facing), and everything in between. So it's worth keeping in mind that e.g. if you're dead set on an ML engineering role there may actually only be 7-10 companies of interest, and this is before any additional filtering for the other things you care about. In addition, although you can land a job at (insert preferred tech company) that Insight partners with, it's worth doing your homework before you apply to see which companies are actually available at which locations. The more traditional tech companies (Google, Facebook, etc.) are mostly only available in Silicon Valley, with more finance/consulting/TV-advertising jobs available in NYC. Seattle/Toronto/Boston have their own flavours too.

The Pros of Insight

There are a few big reasons why Insight was worthwhile. First and foremost, you highly increase your odds of ending up with a great job at a great company that pays 6-figures. Insight's connections are probably better than yours, and your application gets way more exposure and visibility than you otherwise would get. As an example, one member of my cohort reported searching for a job (unsuccessfully) for 6 months prior to attending Insight, while after the program ended they were among the first to be employed. In addition, each company will come to Insight to pitch to you, providing a rare opportunity to connect with companies in person (on your turf!) and ask whatever questions are on your mind.

Second, Insight felt like the real deal in terms of throwing you into the deep end and seeing how you swim. Although you will receive constant feedback from peers/program directors on the 'big picture' of your project, no one is going to do your project for you. It's up to you, and you alone, to get shit done, and there's no real parent/supervisor/safety net to catch you if you fall. Although this can be a primary source of stress at Insight, and certainly has some downsides, I ultimately consider it a "feature not a bug". You learn a lot about yourself, grow faster than you otherwise would, and find the limits of what you're capable of. More than a few times I heard (and share) the sentiment, "if I had this kind of productivity while doing my PhD I would have completed it in half the time". This being said, week 3 was really tough for me and I don't want to go through it again anytime soon (I break into a cold sweat just thinking about it).

Third, once accepted to Insight you really do become part of a family. Coming from academia I didn't have a huge network of connections in Data Science to help with my transition. However, once accepted to Insight I instantly gained access to hundreds of alum who are now working in the field. I have leveraged this network for many purposes from project ideation to interview prep to TN visa advice. Every single Insight alum I've interacted with has been incredibly kind and helpful, and has especially motivated me to "pay it forward" and be as kind and helpful to future Fellows (and non-Fellows!) as I can. Equally as important, you become close with your cohort, bonding over successes and stresses, inside and outside the office (or in our case, partying inside the office after hours (I know, we're lame)). Many members of my cohort I consider to be close friends, and will remain so into the future.

Fourth, it does really seem like the bond with Insight carries into the future. For example, waiting for my visa in Canada was stressful since there was a chance that it wouldn't work out (long story, ask me over a beer). However, the Toronto location of Insight said they were absolutely willing to help me find another job in Canada should my visa not work out, a kind gesture that does not go unappreciated. In addition, while at Insight I saw a few examples of alum being helped by Insight to change jobs (from an existing job that they previously got through the program!). From a monetary perspective, there's no obvious incentive for Insight to do this, as (to my understanding) Insight only gets commission from the initial placement.

Finally, Insight does a great job of rounding out your skills and character in a short amount of time. Surrounded by a cohort of PhDs and program directors in a fast paced environment, you pick up data science skills a lot faster than you otherwise would. You also get a lot of feedback/advice on the "soft skills" that are important to thrive - effective communication, elevator pitches on your background/project, resume tips, salary negotiation, etc. In addition, at Insight there is a "fail fast" mantra that makes you a better version of yourself. Throughout the program you constantly experience forms of rejection - your project idea still sucks, you realize after whiteboarding that you know wayyy less SQL than you thought, you didn't get a callback from Company X, etc. Similar to this guy you become more comfortable with rejection, allowing you to grow more quickly.

How to Best Prepare for Insight

Although you grow a lot at Insight there is only so much you can learn in 7 weeks. For those who have been accepted to the program (congrats!!) but haven't started yet, there's a few key things I'd recommend doing to maximize your chances of success (and if you are thinking of applying to Insight but haven't been accepted yet, I'd still recommend doing all but the first):

  1. Start thinking of project ideas now and bounce them off people who have some combination of tech/business experience. Before I came to Insight I had come up with 6 different project ideas, however I was still very much in an academic mentality. All of the ideas were quickly (and rightfully) rejected by the program directors in the first week, and I found myself having to start again from scratch. Being able to answer:
    • How would my project generate business value for Company X today?
    • How could I generalize my project to generate business value for other companies/sectors?
    are especially important to answer since you'll ultimately be demoing your project to mulitple companies and this question WILL come up.
  2. Learn. Python. Now. I saw a lot of Fellows struggle with their projects because they were simultaneously trying to learn Python and complete their project, making everything take twice as long. Reading a book about Python doesn't count - you need to get as familiar with the popular packages (numpy, pandas, scikit-learn, scipy, etc.) and 'Pythonic' way as you can. Do a mini data science project in Python (and put it on GitHub!) before the program starts. Do coding challenges on leetcode. Ask an experienced Python programmer to give you advice on that triple-nested for loop.
  3. Sharpen up on your machine learning. Broadly speaking, you should know about:
    • The popular supervised/unsupervised algorithms - linear regression, logistic regression, kNN, random forest, recommender systems, LDA, K-means.
    • Best practices and concepts - cross validation, train/test split, bias-variance tradeoff, unbalanced classes, regularization, loss functions, validation, PCA, boosting, bagging, NLP (a whole discipline unto its own).
    More than memorizing an algorithm, it's important to know the pros/cons of these algorithms and methods. A few examples:
    • When would I use logistic regression over random forest?
    • When would I use an L2 over L1 regularization?
    • When would I train a generative model over a discriminative one?
    After the project phase of Insight you basically get 2 weeks of interview prep before interviews start, and the more of these concepts you're unfamiliar with the more you'll end up just cramming/memorizing vs. genuinely understanding. Regardless of the specific role you want in Data Science, knowledge of machine learning is a requirement (unlike e.g. SQL), and especially for the more tech-heavy roles the interviewers will know if you just memorized an algorithm yesterday.
  4. Work on your communication skills. Getting a job at Company X largely depends on your ability to explain your project and background to non-experts. If you're a really smart person but don't know how to communicate ideas in a simple manner you are unlikely to get the job. It's also important to get some kind of outside confirmation that you're a good communicator. Most people think they're good communicators, but it's generally a rare skill. You're also going to have to demo your project in a public speaking kind of environment, i.e. at the front of the room to an audience of people. So, give a public lecture on your research AND get feedback on it. If you're still in academia, sign up to lead the weekly arXiv paper discussion AND get feedback on it. Try explaining your PhD research to your grandmother and her friends AND get feedback on it.