Policy:User-Agent policy/ja: Difference between revisions

From Wikimedia Foundation Governance Wiki
Content deleted Content added
FuzzyBot (talk | contribs)
Updating to match new version of source page
FuzzyBot (talk | contribs)
Updating to match new version of source page
Line 26: Line 26:
</div>
</div>
<pre>
<pre>
User-Agent: CoolTool/0.0 (https://example.org/cool-tool/; cool-tool@example.org) generic-library/0.0
User-Agent: CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org) generic-library/0.0
</pre>
</pre>


Line 38: Line 38:


<div lang="en" dir="ltr" class="mw-content-ltr">
<div lang="en" dir="ltr" class="mw-content-ltr">
Do not copy a browser's user agent for your bot, as bot-like behavior with a browser's user agent will be assumed malicious.<ref>[//lists.wikimedia.org/pipermail/wikitech-l/2010-February/046783.html [Wikitech-l&#93; User-Agent:]</ref> Do not use generic agents such as "curl", "lwp", "Python-urllib", and so on. For large frameworks like pywikibot, there are so many users that just "pywikibot" is likely to be somewhat vague. Including detail about the specific task/script/etc would be a good idea, even if that detail is opaque to anyone besides the operator.<ref>{{cite web|url=http://lists.wikimedia.org/pipermail/mediawiki-api/2014-July/003308.html|title=Clarification on what is needed for "identifying the bot" in bot user-agent?|publisher=Mediawiki-api|author=Anomie|date=31 July 2014}}</ref>
Do not copy a browser's user agent for your bot, as bot-like behavior with a browser's user agent will be assumed malicious.<ref>[//lists.wikimedia.org/pipermail/wikitech-l/2010-February/046783.html [Wikitech-l&#93; User-Agent:]</ref> Do not use generic agents such as "curl", "lwp", "Python-urllib", and so on. For large frameworks like pywikibot, there are so many users that just "pywikibot" is likely to be somewhat vague. Including detail about the specific task/script/etc would be a good idea, even if that detail is opaque to anyone besides the operator.<ref>{{cite web|url=https://lists.wikimedia.org/pipermail/mediawiki-api/2014-July/003308.html|title=Clarification on what is needed for "identifying the bot" in bot user-agent?|publisher=Mediawiki-api|author=Anomie|date=31 July 2014}}</ref>
</div>
</div>


Line 58: Line 58:
<div lang="en" dir="ltr" class="mw-content-ltr">
<div lang="en" dir="ltr" class="mw-content-ltr">
On Wikimedia wikis, if you don't supply a <code>User-Agent</code> header, or you supply an empty or generic one, your request will fail with an HTTP 403 error. Other MediaWiki installations may have similar policies.
On Wikimedia wikis, if you don't supply a <code>User-Agent</code> header, or you supply an empty or generic one, your request will fail with an HTTP 403 error. Other MediaWiki installations may have similar policies.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== JavaScript ===
</div>
</div>


Line 67: Line 71:
// Using XMLHttpRequest
// Using XMLHttpRequest
xhr.setRequestHeader( 'Api-User-Agent', 'Example/1.0' );
xhr.setRequestHeader( 'Api-User-Agent', 'Example/1.0' );
</syntaxhighlight>

<syntaxhighlight lang="javascript">
// Using jQuery
// Using jQuery
$.ajax( {
$.ajax( {
url: remoteUrlWithOrigin,
url: 'https://example/...',
data: queryData,
data: ...,
dataType: 'json',
dataType: 'json',
type: 'POST',
type: 'GET',
headers: { 'Api-User-Agent': 'Example/1.0' },
headers: { 'Api-User-Agent': 'Example/1.0' },
success: function(data) {
} ).then( function ( data ) {
// do something with data
// ..
}
} );
} );
</syntaxhighlight>

<syntaxhighlight lang="javascript">
// Using mw.Api, specify it when creating the mw.Api object
// Using mw.Api
var api = new mw.Api( {
var api = new mw.Api( {
ajax: {
ajax: {
Line 86: Line 91:
}
}
} );
} );
api.get( {...} ).done(function(data) {
api.get( ... ).then( function ( data ) {
// do something with data
// ...
});
});
</syntaxhighlight>

<syntaxhighlight lang="javascript">
// Using fetch
// Using Fetch
fetch( remoteUrlWithOrigin, {
fetch( 'https://example/...', {
method: 'POST',
method: 'GET',
headers: new Headers( {
headers: new Headers( {
'Api-User-Agent': 'Example/1.0'
'Api-User-Agent': 'Example/1.0'
} )
} )
// Other init settings such as 'credentials'
} ).then( function ( response ) {
} ).then( function ( response ) {
if ( response.ok ) {
return response.json();
return response.json();
}
throw new Error( 'Network response was not ok: ' + response.statusText );
} ).then( function ( data ) {
} ).then( function ( data ) {
// do something with data
// ...
});
});
</syntaxhighlight>
</syntaxhighlight>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== PHP ===
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
<div lang="en" dir="ltr" class="mw-content-ltr">
Line 112: Line 118:


<syntaxhighlight lang="php">
<syntaxhighlight lang="php">
ini_set('user_agent', 'MyCoolTool/1.1 (https://example.org/MyCoolTool/; MyCoolTool@example.org) BasedOnSuperLib/1.4');
ini_set( 'user_agent', 'CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org)' );
</syntaxhighlight>
</syntaxhighlight>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== cURL ===
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
<div lang="en" dir="ltr" class="mw-content-ltr">
Line 120: Line 130:


<syntaxhighlight lang="php">
<syntaxhighlight lang="php">
curl_setopt($curl, CURLOPT_USERAGENT, 'MyCoolTool/1.1 (https://example.org/MyCoolTool/; MyCoolTool@example.org) BasedOnSuperLib/1.4');
curl_setopt( $curl, CURLOPT_USERAGENT, 'CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org)' );
</syntaxhighlight>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Python ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In Python, you can use the [[wikipedia:Requests_(software)|Requests]] library to set a header:
</div>

<syntaxhighlight lang="python">
import requests

url = 'https://example/...'
headers = {'User-Agent': 'CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org)'}

response = requests.get(url, headers=headers)
</syntaxhighlight>
</syntaxhighlight>



Revision as of 14:58, 9 April 2022

ウィキメディアのサイト群では2010年2月15日付でリクエストごとにHTTP ユーザーエージェント ヘッダ が必要になりました。技術職員による運用上の決定であり、技術メーリングリストで発表と議論をしています[1][2]。根本原理としてユーザーエージェントのストリングを送信しないクライアントとは、そのほとんどが悪意のあるスクリプトでサーバに過剰な負担をかけ、プロジェクトの益にならないものです。ただしユーザーエージェントのストリングの非説明的既定値、すなわち Perl の libwww の使用分なども、ウィキメディアのウェブサイト (もしくは api.php などウェブサイトの一部) を利用するとブロックされる場合があります。

ユーザーエージェント (ブラウザもしくはスクリプト) からユーザーエージェント ヘッダを送信しない場合、次のようなエラー警告が表示される場合があります。

スクリプトは有効な情報を含むユーザーエージェントのストリングとして連絡先情報を使用するべきで、未使用の場合は予告なく IP ブロックの対象となる場合があります。

ユーザーエージェントから送信されたユーザーエージェント ヘッダがブラックリストに含まれる場合 (有益か無益かに関わらず「lwp」で始まるユーザーエージェント ストリングなど) は、 次のような状態を説明していない (偽の) エラー警告が表示される場合があります。

現在、このサーバーには技術的な問題が発生しています。一時的な現象であり、早急に対応する必要があります。数分後に再度、ご利用ください。

This change is most likely to affect scripts (bots) accessing Wikimedia websites such as Wikipedia automatically, via api.php or otherwise, and command line programs.[3] If you run a bot, please send a User-Agent header identifying the bot with an identifier that isn't going to be confused with many other bots, and supplying some way of contacting you (e.g. a userpage on the local wiki, a userpage on a related wiki using interwiki linking syntax, a URI for a relevant external website, or an email address), e.g.:

User-Agent: CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org) generic-library/0.0

The generic format is <client name>/<version> (<contact information>) <library/framework name>/<version> [<library name>/<version> ...]. Parts that are not applicable can be omitted.

If you run an automated agent, please consider following the Internet-wide convention of including the string "bot" in the User-Agent string, in any combination of lowercase or uppercase letters. This is recognized by Wikimedia's systems, and used to classify traffic and provide more accurate statistics.

Do not copy a browser's user agent for your bot, as bot-like behavior with a browser's user agent will be assumed malicious.[4] Do not use generic agents such as "curl", "lwp", "Python-urllib", and so on. For large frameworks like pywikibot, there are so many users that just "pywikibot" is likely to be somewhat vague. Including detail about the specific task/script/etc would be a good idea, even if that detail is opaque to anyone besides the operator.[5]

Web browsers generally send a User-Agent string automatically; if you encounter the above error, please refer to your browser's manual to find out how to set the User-Agent string. Note that some plugins or proxies for privacy enhancement may suppress this header. However, for anonymous surfing, it is recommended to send a generic User-Agent string, instead of suppressing it or sending an empty string. Note that other features are much more likely to identify you to a website — if you are interested in protecting your privacy, visit the Panopticlick project.

Browser-based applications written in JavaScript are typically forced to send the same User-Agent header as the browser that hosts them. This is not a violation of policy, however such applications are encouraged to include the Api-User-Agent header to supply an appropriate agent.

2015年現在、技術的に強制されたユーザーエージェント必須条件はないものの、必要に応じて個別の事例で強制される場合があります。

[6]

Code examples

On Wikimedia wikis, if you don't supply a User-Agent header, or you supply an empty or generic one, your request will fail with an HTTP 403 error. Other MediaWiki installations may have similar policies.

JavaScript

If you are calling the API from browser-based JavaScript, you won't be able to influence the User-Agent header: the browser will use its own. To work around this, use the Api-User-Agent header:

// Using XMLHttpRequest
xhr.setRequestHeader( 'Api-User-Agent', 'Example/1.0' );
// Using jQuery
$.ajax( {
    url: 'https://example/...',
    data: ...,
    dataType: 'json',
    type: 'GET',
    headers: { 'Api-User-Agent': 'Example/1.0' },
} ).then( function ( data )  {
    // ..
} );
// Using mw.Api
var api = new mw.Api( {
    ajax: {
        headers: { 'Api-User-Agent': 'Example/1.0' }
    }
} );
api.get( ... ).then( function ( data ) {
    // ...
});
// Using Fetch
fetch( 'https://example/...', {
    method: 'GET',
    headers: new Headers( {
        'Api-User-Agent': 'Example/1.0'
    } )
} ).then( function ( response ) {
    return response.json();
} ).then( function ( data ) {
    // ...
});

PHP

In PHP, you can identify your user-agent with code such as this:

ini_set( 'user_agent', 'CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org)' );

cURL

Or if you use cURL:

curl_setopt( $curl, CURLOPT_USERAGENT, 'CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org)' );

Python

In Python, you can use the Requests library to set a header:

import requests

url = 'https://example/...'
headers = {'User-Agent': 'CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org)'}

response = requests.get(url, headers=headers)

注記

  1. The Wikitech-l 2010年2月話題ごとの過去ログ
  2. User-Agent: | Wikipedia | Wikitech
  3. API:FAQ - MediaWiki
  4. [Wikitech-l] User-Agent:
  5. Anomie (31 July 2014). "Clarification on what is needed for "identifying the bot" in bot user-agent?". Mediawiki-api. 
  6. gmane.science.linguistics.wikipedia.technical/83870 (deadlink)

See also