Python crawler advanced must-have | Logical analysis of the encryption parameters of the jump link on a certain information publishing platform

Today website

aHR0cDovLzIyMy40LjY1LjEzMTo4MDgwLyMvY29tcGFueQ==

This website comes from the technical exchange group of Xianyu

Packet capture analysis

The structure of this website is very simple, the information of the list page can be directly obtained through the interface, let's take a look

Data can be returned by directly requesting the interface.

The content we need to analyze is the link encryption of the details page, as shown below

Hover the mouse over the blue text and you can see that there is a CORPCODE parameter after the link (PS: Actually it doesn’t matter whether this parameter is or not, because the dog head is not used for request data)

Encrypted positioning

It can be found directly with the Initiator of the network panel

Click on the blue link at the back to jump directly

The location of the jump is where the request was made

Here you can see that the parameter that needs to be analyzed is t

It is generated by the following string of expressions

Object(m.b)(e.CorpCode)

So just hit a breakpoint and debug it

The e here is the content displayed on the list page, which can be obtained from the interface of the list page

The parameter e.CorpCode here becomes ciphertext after mb, so the logic of mb can be realized

Enter the logic of analyzing mb, you can see the clear encryption logic

The three elements of AES: secret key, padding, offset

All have been displayed in the code, get it on the website to verify

You can see that the encryption is the same as on the page.

Just copy directly in Python, or directly import the encryption library of NodeJS.

However, this parameter is not required for the details page data acquisition, so just take it for fun~