Import.io delivers solutions that enables users to convert the mass of data on web sites into structured, machine-readable data with no coding required. Our highly efficient and scalable platform allows customers to process millions of URLs and gain access to billions of rows of data that they use for mission critical use cases. Some common uses include building training sets for machine learning models, doing retail price monitoring and analysis, web data integration into business applications, market research, academic research, etc.
The Role:
We are looking to build world class capabilities to extract data from any source of data that is connected to the internet. On the most part, that means websites. And that’s where you come in.
We’re looking for you – the expert who can automate navigating any website to get to the data we are looking for. Of course that probably means using a browser and browser automation tools. But there are probably many other ways you can think of to accomplish the mission, and you know which approach makes sense. To accomplish this, you will have the opportunity to work with our architects and CTO.
Types of challenges you’ll tackle on at Import.io
We are a tight knit group of high calibre engineers. So while we can write down some examples of types of things you will be doing, expect to have a much wider range of tasks. After all, that’s part of the reason why you’d want to join a small company, right? Don’t get stuck only looking at a narrow slice of the pie at a big company!
At Import.io, Architects are looked to as thought leaders and trusted advisors by our entire company including executive staff, engineers, sales, and ultimately our customers. Architects are expected to be subject matter experts in the area/categories that you own – architecture, technology to use, integration, security, business strategy, etc.
The primary responsibilities of this role include:
- Use your experience and knowledge to build a browser automation framework that allows users to programmatically navigate any website and get to the data we are looking for.
- That means you’re a headless browser and automation wizard. You probably worked with tools like Puppeteer, Selenium, and many other automation tools.
- You know intimately how websites are constructed. Given that, you know how to de-construct it. Often times, the first thing you feel like doing when you load the page is look under the hood using the Developer console. You immediately want to inspect the DOM and pay around with the public APIs.
- You know how to create this so it can scale up/down on its own depending on load. We are an AWS shop and use kubernetes (k8s). You should have extensive experience with both (or similar technologies like mesos, Google Cloud, Microsoft Azure)
- You know how to instrument the platform, so at any given moment, you can tell what’s going on. You also know now to rely on “looking” to find out something is wrong because you have a keen sense for effective use of monitoring and alerts. Tools like PagerDuty and OpsGenie come to mind.
- And by the way, you HATE it when you get a PagerDuty calls you. Each time it does, you take a personal and do whatever it takes to make sure you never get called again.
- Figure out the best ways for Import.io’s web data integration platform get around anti-bot, avoid our traffic going to Google Analytics, and work around all other anti-crawling technologies.
We’re not here to tell you what skills are required. We can, but better would be for you to tell us what skills you have, and why you think you can accomplish the above. But to help give you an idea what we have in mind:
- Must be a self-starter with the ability and drive to own and deliver.
- You have in depth knowledge of networking protocols, especially HTTP and other relevant networking technologies.
- Expertise in browsers and other client technologies.
- While you obsess over knowing the latest technologies, you have the business acumen to know when it should be used versus when to stick to older, tried and true technologies.
- You have a professional attitude. You have the ability to analyse complex technical issues and provide options to overcome challenges. That also means you can debate merits and pitfalls with peers then commit and align to the decision.
- You don’t need babysitting. You can execute with a sense of urgency and with minimum oversight. After all, others are going to be looking to you for guidance, not the other way around.
- You hate it when the code you are responsible for doesn’t work. You take it personal. You do what it takes for it to never happen again.
- That means you know that YOU own the quality of the product. You don’t need to rely on QA
- To do that, you know how to develop testing plans, validate against design goals, and measure quality objectively via metrics.
- You want to, and know how to, automate everything. You never want (or need) to manually do the same thing twice.
- You are your own biggest critic. You get pissed off at yourself when you can’t finish something you started and deploy it all the way to production.
- Extensive experience in the software industry, with extensive knowledge of software development and enterprise software technologies.
Apply to: [email protected]