Let's talk about the scenario first. You need to write a crawler to get data from a certain interface address regularly, and then store it in the local database.
Therefore, we need something that can be executed regularly. In the 5fire's knowledge system, apart from executing a stored procedure or SQL statement in the database, there is no concept of how to execute a task in the system.
So I asked my colleagues. I know that in ubuntu, to be precise, in linux, there is such a thing-crontab, which is specially used to customize the execution of tasks. Give it a brief explanation and it will be used.
Although this thing is relatively simple, in order to ensure the integrity of the knowledge system, we still have to look up what crontab is.
The word crontab means: timed task.
Look at the encyclopedia definition [full definition]:
The crontab command is commonly used in Unix and Unix-like operating systems to set instructions to be executed periodically. This command reads instructions from the standard input device and stores them in the "crontab" file for later reading and execution. The word comes from the Greek chronos (χρόνος), which originally means time.
Usually, the instructions stored in crontab are activated by the daemon, and crond often runs in the background, checking every minute whether there is a scheduled job to be executed. Such jobs are generally called cron jobs.
With the above concepts, the use of crontab will be clearer.
Since the system needs to be checked every minute, there must be a basis for checking, such as a configuration file or something.
Or take a look at the Wiki:
The crontab file contains a series of jobs and instructions sent to the cron daemon. Each user can have his own crontab file; at the same time, the operating system saves a crontab file for the entire system, which is usually stored in a subdirectory under /etc or /etc, and this file can only be used by the system administrator modify.
Each line of the crontab file follows a specific format, separated into several fields by spaces or tabs, and each field can be placed with a single or multiple values.
Okay, let's start using it. It is estimated that some people know how to use it from the definition. But I still want to record.
Steps for usage:
Run crontab -e on the terminal [Explanation: Edit the configuration file]
Choose the editor you want to use, most people will choose vi.
At this time the configuration file has been opened, just write the configuration according to his format.
Well, it's as simple as I think. .
for example:
There is a python script in my home directory, helloworld.py
#coding:utf-8
print 'hello world by crontab!'
I want this script to be executed at 7:30 in the morning every day.
Therefore, the crontab configuration file for this task is:
# m h dom mon dow command
307*** python /home/the5fire/testcrontab.py >>/home/the5fire/testcrontab.log 2>&1
To explain briefly, the meaning of this configuration is to run the testcrontab.py file in my home directory with python at 7:30 every day, and output the output to testcrontab.log. The latter 2>&1 means error The output of is also output to standard output (2 means error, 2> means error output, & means equal, 1 means correct), so if there is an error in operation, the error will be output to the previously defined log.
In addition, there is something to say about the appropriate execution of the command.
The above is just a few timings to execute, so how do I set it to execute at a certain frequency. For example, it is executed sequentially every minute.
The corresponding configuration is
# m h dom mon dow command
*/1**** python /home/the5fire/testcrontab.py >>/home/the5fire/testcrontab.log 2>&1
Let's come to another scenario. How to write the configuration that I want to execute every 3 minutes between 6 am and 8 am every day:
# m h dom mon dow command
*/36-8*** python /home/the5fire/testcrontab.py >>/home/the5fire/testcrontab.log 2>&1
So far, it should be used. The five asterisks indicate different execution units (minute, hour, day, month, year), and the backslash indicates frequency. *- *
Recommended Posts