利用爬虫爬取5万条城市评论并对其进行情感分析

# tough-cookie [RFC 6265](https://tools.ietf.org/html/rfc6265) Cookies and CookieJar for Node.js [](https://nodei.co/npm/tough-cookie/) [](https://travis-ci.org/salesforce/tough-cookie) ## Synopsis ```javascript var tough = require("tough-cookie"); var Cookie = tough.Cookie; var cookie = Cookie.parse(header); cookie.value = "somethingdifferent"; header = cookie.toString(); var cookiejar = new tough.CookieJar(); // Asynchronous! var cookie = await cookiejar.setCookie( cookie, "https://currentdomain.example.com/path" ); var cookies = await cookiejar.getCookies("https://example.com/otherpath"); // Or with callbacks! cookiejar.setCookie( cookie, "https://currentdomain.example.com/path", function (err, cookie) { /* ... */ } ); cookiejar.getCookies("http://example.com/otherpath", function (err, cookies) { /* ... */ }); ``` Why the name? NPM modules `cookie`, `cookies` and `cookiejar` were already taken. ## Installation It's _so_ easy! Install with `npm` or your preferred package manager. ```sh npm install tough-cookie ``` ## Node.js Version Support We follow the [node.js release schedule](https://github.com/nodejs/Release#release-schedule) and support all versions that are in Active LTS or Maintenance. We will always do a major release when dropping support for older versions of node, and we will do so in consultation with our community. ## API ### tough The top-level exports from `require('tough-cookie')` can all be used as pure functions and don't need to be bound. #### `parseDate(string)` Parse a cookie date string into a `Date`. Parses according to [RFC 6265 Section 5.1.1](https://datatracker.ietf.org/doc/html/rfc6265#section-5.1.1), not `Date.parse()`. #### `formatDate(date)` Format a `Date` into an [RFC 822](https://datatracker.ietf.org/doc/html/rfc822#section-5) string (the RFC 6265 recommended format). #### `canonicalDomain(str)` Transforms a domain name into a canonical domain name. The canonical domain name is a domain name that has been trimmed, lowercased, stripped of leading dot, and optionally punycode-encoded ([Section 5.1.2 of RFC 6265](https://datatracker.ietf.org/doc/html/rfc6265#section-5.1.2)). For the most part, this function is idempotent (calling the function with the output from a previous call returns the same output). #### `domainMatch(str, domStr[, canonicalize=true])` Answers "does this real domain match the domain in a cookie?". The `str` is the "current" domain name and the `domStr` is the "cookie" domain name. Matches according to [RFC 6265 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc6265#section-5.1.3), but it helps to think of it as a "suffix match". The `canonicalize` parameter toggles whether the domain parameters get normalized with `canonicalDomain` or not. #### `defaultPath(path)` Given a current request/response path, gives the path appropriate for storing in a cookie. This is basically the "directory" of a "file" in the path, but is specified by [Section 5.1.4 of the RFC](https://datatracker.ietf.org/doc/html/rfc6265#section-5.1.4). The `path` parameter MUST be _only_ the pathname part of a URI (excluding the hostname, query, fragment, and so on). This is the `.pathname` property of node's `uri.parse()` output. #### `pathMatch(reqPath, cookiePath)` Answers "does the request-path path-match a given cookie-path?" as per [RFC 6265 Section 5.1.4](https://datatracker.ietf.org/doc/html/rfc6265#section-5.1.4). Returns a boolean. This is essentially a prefix-match where `cookiePath` is a prefix of `reqPath`. #### `parse(cookieString[, options])` Alias for [`Cookie.parse(cookieString[, options])`](#cookieparsecookiestring-options). #### `fromJSON(string)` Alias for [`Cookie.fromJSON(string)`](#cookiefromjsonstrorobj). #### `getPublicSuffix(hostname)` Returns the public suffix of this hostname. The public suffix is the shortest domain name upon which a cookie can be set. Returns `null` if the hostname cannot have cookies set for it. For example: `www.example.com` and `www.subdomain.example.com` both have public suffix `example.com`. For further information, see the [Public Suffix List](http://publicsuffix.org/). This module derives its list from that site. This call is a wrapper around [`psl`](https://www.npmjs.com/package/psl)'s [`get` method](https://www.npmjs.com/package/psl##pslgetdomain). #### `cookieCompare(a, b)` For use with `.sort()`, sorts a list of cookies into the recommended order given in step 2 of ([RFC 6265 Section 5.4](https://datatracker.ietf.org/doc/html/rfc6265#section-5.4)). The sort algorithm is, in order of precedence: - Longest `.path` - oldest `.creation` (which has a 1-ms precision, same as `Date`) - lowest `.creationIndex` (to get beyond the 1-ms precision) ```javascript var cookies = [ /* unsorted array of Cookie objects */ ]; cookies = cookies.sort(cookieCompare); ``` > **Note**: Since the JavaScript `Date` is limited to a 1-ms precision, cookies within the same millisecond are entirely possible. This is especially true when using the `now` option to `.setCookie()`. The `.creationIndex` property is a per-process global counter, assigned during construction with `new Cookie()`, which preserves the spirit of the RFC sorting: older cookies go first. This works great for `MemoryCookieStore` since `Set-Cookie` headers are parsed in order, but is not so great for distributed systems. Sophisticated `Store`s may wish to set this to some other _logical clock_ so that if cookies A and B are created in the same millisecond, but cookie A is created before cookie B, then `A.creationIndex < B.creationIndex`. If you want to alter the global counter, which you probably _shouldn't_ do, it's stored in `Cookie.cookiesCreated`. #### `permuteDomain(domain)` Generates a list of all possible domains that `domainMatch()` the parameter. Can be handy for implementing cookie stores. #### `permutePath(path)` Generates a list of all possible paths that `pathMatch()` the parameter. Can be handy for implementing cookie stores. ### Cookie Exported via `tough.Cookie`. #### `Cookie.parse(cookieString[, options])` Parses a single Cookie or Set-Cookie HTTP header into a `Cookie` object. Returns `undefined` if the string can't be parsed. The options parameter is not required and currently has only one property: - _loose_ - boolean - if `true` enable parsing of keyless cookies like `=abc` and `=`, which are not RFC-compliant. If options is not an object it is ignored, which means it can be used with [`Array#map`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map). To process the Set-Cookie header(s) on a node HTTP/HTTPS response: ```javascript if (Array.isArray(res.headers["set-cookie"])) cookies = res.headers["set-cookie"].map(Cookie.parse); else cookies = [Cookie.parse(res.headers["set-cookie"])]; ``` _Note:_ In version 2.3.3, tough-cookie limited the number of spaces before the `=` to 256 characters. This limitation was removed in version 2.3.4. For more details, see [issue #92](https://github.com/salesforce/tough-cookie/issues/92). #### Properties Cookie object properties: - _key_ - string - the name or key of the cookie (default `""`) - _value_ - string - the value of the cookie (default `""`) - _expires_ - `Date` - if set, the `Expires=` attribute of the cookie (defaults to the string `"Infinity"`). See `setExpires()` - _maxAge_ - seconds - if set, the `Max-Age=` attribute _in seconds_ of the cookie. Can also be set to strings `"Infinity"` and `"-Infinity"` for non-expiry and immediate-expiry, respectively. See `setMaxAge()` - _domain_ - string - the `Domain=` attribute of the cookie - _path_ - string - the `Path=` of the cookie - _secure_ - boolean - the `Secure` cookie flag - _httpOnly_ - boolean - the `HttpOnly` cookie