Step 9: Add error handling - your applications should not crash if they encounter a data-related exception (TypeError, Inde圎rror, etc.) but instead write it to a log file (as a minimum, print it to the console) and continue running. Step 8: Write a consumer application which runs 24/7 awaiting for something to appear in the queue, and when it does, it calls your rest api from step 2 using the data received from the queue, adds the returned result (predicted price, or whatever) to the data, then runs some validation / cleaning on the data, and saves it in some database (SQLite is the easiest to have running on your local computer) using an ORM (such as SQLalchemy). Step 7: read the closest thing in existence to being the data engineering book:
#Bitcraft reddit how to#
Step 6: Learn how to use cron (for bonus points: Airflow) to make the application from step 5 automatically run every second day at 8 am I would strongly recommend sending each item/record as a separate queue message instead of sending them all as one huge message. Step 5: Write a publisher application which reads a csv, xml, or json file from disk (for bonus points: from someone else's public REST API for data, for example ), and turns the data contained within into a list of python dictionaries or serializable objects (btw, read up on serializing, it's important), and then sends the results into a kafka or rabbitMQ queue. Step 4 (optional, but recommended): Learn the basics of Java (this tutorial should be more than enough ) and read
#Bitcraft reddit free#
If using Python, I highly recommend using P圜harm (the free community edition is enough) over Jupyter or Anaconda, the latter will let you do many bad things which would trigger a red warning in P圜harm (such us doing import in the middle of the file). Step 3: Write a REST API which takes arguments from the URL, uses these arguments to run some predictive model of your creation, and then returns the result since you already know Python, I'd recommend using Flask, there are many free tutorials, just google it. This is what you’ve said already and I’m trying to show you why it looks bad in an interview setting. I don’t think anybody would hire somebody who shows a pattern of missing deadlines and blaming others. I’m not saying the original people weren’t at fault, but the act of blaming another person is the issue.īlaming others in an interview or business setting is a huge red flag.
#Bitcraft reddit code#
Nobody needs to hear the code is broken because they already know it. įinally, you are coming back at me blaming others again when you stated everyone already knew the original code was broken. The book “the pragmatic programmer” is a fantastic read for balancing project deadlines and code. If something works within specifications, then it might be best to just make notes in a comment to “make it better” later. The best, most correct solution isn’t always the best and dragging a project past deadlines because it’s not perfect is bad for business. If your company has a pattern of pushing bad code, then the solution might be better code reviews. If you have to, drop everything when code review comes up so you have a solution in hand when you raise an objection. Don’t say there is a problem with a persons ideas unless you can justify your changes immediately.
0 kommentar(er)
