Here is an example that shows data streaming and loading data at once. In Anaconda’s local directory, you can create a text file. Add some data to it. You can use it to understand the concept.
IN THIS PAGE
- Getting Data at once into Memory
- Python logic to get data at once to memory
- Commands used
- Streaming data
Getting Data at once into Memory
When you take the entire data into memory, your program may hang if your system lacks memory. It then causes you to restart the process is a time-consuming task. So the point you need to note here is loading data at once may throw performance issues.
Python logic to get data at once to memory
The sample.py script has python code. Its functionality is to open the file in read mode and get the data at once. This is the best example to tell in interviews. I am giving this here for your understanding.
- nano sample.py
Here I have used for-loop and get records one by one. The method, you can say data streaming. The stream.py has for-loop logic added. Which reads records (data) one by one.
The output you can see is displayed the file records one by one.
With the above example, you have got a clear idea of streaming. It’s helpful for data science engineers who use Python for analysis.