Public Beta Launch
Announcing the launch of our first public test
Welcome to the all-new Optimus Mine blog. Here we will be regularly updating you with what’s happening at Optimus Mine and what new features you can look forward to. But now without further ado let’s get into the most important point: the launch of our first public beta!
Like so many startups we’ve been faced with the hard decision of what features to include, as there are many on our roadmap. Our ultimate vision is to enable non-programmers to automate the collection, processing and warehousing of large amounts of data from the web, but for our initial release we decided to strip out all the non-essential UI that would make Optimus Mine easy to use for entry-level users, and to give our undivided attention to the core features for data analysts, developers and tech-savvy entrepreneurs, focusing on great scalability and a wide array configuration options.
Using Optimus Mine today you can automate the process of extracting, transforming and warehousing data, reducing the time you spend doing tedious, time consuming tasks.
Getting started
First things first - head over to our downloads page to download & install the latest version of Optimus Mine. Once you’ve completed that, read on to get started with our first three tutorials.
Scrape an RSS feed to JSON
The first tutorial is about Using Optimus Mine to scrape an RSS feed. The goal of this tutorial is to introduce you to the basic concepts of Optimus Mine, as well as the most common configuration parameters.
Extracting Companies House data and storing it in BigQuery
The second tutorial teaches you how to extract data from Companies House, the UK business register, and upload the results to Google BigQuery. In this tutorial you learn a bit more on how the export features work in Optimus Mine, saving you time when collecting, processing and warehousing of large amounts of data.
Download & transform multiple XML files to a single CSV
In the third tutorial we focus on data transformation, downloading hundreds of XML files from the web and extracting the data into a single csv. As most big data warehouses do not ingest all formats, we wanted to show you how using Optimus Mine you can collect and transform hundreds of files in just a few minutes.
What’s next?
Once you have completed the tutorials and feel confident to get working on a data pipeline of your own, we’re happy to help all our beta testers with FREE support for the duration of the beta test. Simply email any questions you might have to support@optimusmine.com and we will do our best to resolve your issues within 24h.
Also, we’re really keen to hear your feedback on both Optimus Mine and the tutorials so feel free to reach out with your comments.
Lastly, if there are any features requests you have for Optimus Mine, please submit them at https://optimusmine.com/contact/support/. We’re always trying to improve and would be delighted to hear your suggestion!
Thanks for reading! Time to extract, transform and roll out!