[Date Prev][Date Next][Thread Prev][][Date Index][Thread Index]

Re: Changing a user agent string

On 2018-01-15 10:44, Katsumi Yamaoka wrote:
> In [emacs-w3m:12872]
> On Sun, 14 Jan 2018 00:44:01 -0500, Boruch Baum wrote:
> > I wanted to easily see how pages might appear differently using
> > different user-agent strings, so I wrote the following. If there is
> > interest to add it to the project, I would change the alist to a
> > defcustom.
> I sometimes did it.  So, I think such a feature would be helpful.
> But if we implement the feature, I think it would be enough that
> changing the user-agent field temporarily, not permanently.  For
> example, `C-u C-u R' will prompt a user to select a user-agent
> string, reload the current page with it, and restore the value
> of `w3m-user-agent'.

Attached is the patch.

1] The feature is currently implemented only for `w3m-reload-this-page',
   but shouldn't it somehow also implemented for `w3m-goto-url',
   `w3m-goto-url-new-session', and maybe even `w3m-reload-all'pages'?

   1.1] Do you want it done the same way, using just `C-u'? Or `C-u C-u'
        in order to keep it consistent? I'm not sure what would be less
        confusing to users.

   1.2] Are there other functions that should implement this feature?

2] I came across an "issue" in that w3m seems to insist on ALWAYS
   sending a user-agent string. My work-around was to push a "?" as a
   dummy user-agent string when `w3m-add-user-agent' is nil. However,
   this itself may be identifiable. Maybe an alternative work-around
   would be better? Possibly "error" or "n/a".

   2.1] I'm considering submitting a bug report to w3m. However, I
        suspect that it would carry more clout if it came from the
        emacs-w3m project rather than from  me.

3] When looking at item #2, I came across `w3m-add-referer'. Do I
   understand correctly that the default always suppresses referers when
   using https? Now that the project is encouraging https as a default,
   should that behavior continue?

4] I added an option for three prefix arguments, ie `C-u C-u C-u' to
   cover the cases of someone wanting a new user-agent string but not
   necessarily wanting to clear form and post data.

5] The feature is coded in such a way that when called directly
   interactively, it will permanently change the user-agent string, but
   otherwise the change will only apply for the current operation.

CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0
Index: ChangeLog
RCS file: /storage/cvsroot/emacs-w3m/ChangeLog,v
retrieving revision 1.3639
diff -u -r1.3639 ChangeLog
--- ChangeLog	16 Jan 2018 09:35:31 -0000	1.3639
+++ ChangeLog	17 Jan 2018 17:02:36 -0000
@@ -1,3 +1,10 @@
+2018-01-17  Boruch Baum  <boruch_baum@xxxxxxx>
+	* w3m.el (w3m-user-agent-change): New feature.
+	(w3m-reload-this-page): Integrate new feature.
+	(w3m-request-arguments, w3m-header-arguments): Ensure w3m and
+	w3mmee honor `w3m-add-user-agent'.
 2018-01-16  Katsumi Yamaoka  <yamaoka@xxxxxxx>
 	Make `cd doc; make pdf' work again
Index: w3m.el
RCS file: /storage/cvsroot/emacs-w3m/w3m.el,v
retrieving revision 1.1695
diff -u -r1.1695 w3m.el
--- w3m.el	13 Dec 2017 04:01:49 -0000	1.1695
+++ w3m.el	17 Jan 2018 17:02:38 -0000
@@ -319,6 +319,23 @@
   :group 'w3m
   :type 'boolean)
+(defvar w3m-user-agent-default-alist (list
+  (cons "Emacs-w3m (user default)" w3m-user-agent)
+  (cons "Emacs-w3m (package default)" (concat "Emacs-w3m/" emacs-w3m-version " " w3m-version))
+  '("Don't send user agent" . ""))
+"An default alist of user agent strings, used when offering the user the opportunity to change user agent strings. This should normally not be modified; instead modify `w3m-user-agent-alist'.")
+(defcustom w3m-user-agent-alist '(
+  ("Android Webkit" . "Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30")
+  ("Firefox 57" . "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0")
+  ("IE Mobile" . "Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0)")
+  ("Opera Mini 9.80" . "Opera/9.80 (J2ME/MIDP; Opera Mini/9.80 (S60; SymbOS; Opera Mobi/23.348; U; en) Presto/2.5.25 Version/10.54")
+  ("Opera Mobile 12" . "Opera/12.02 (Android 4.1; Linux; Opera Mobi/ADR-1111101157; U; en-US) Presto/2.9.201 Version/12.02")
+  ("Chrome 51" . "Mozilla/5.0 (Linux; Android 5.1.1; VS810PP Build/LMY47V) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.81 Mobile Safari/537.36"))
+"An alist of user agent strings. Each entry should be a cons of a short descriptive string and the user agent string."
+:group 'w3m
+:type '(repeat (cons (string :tag "Short Description") (string :tag "User Agent string"))))
 (defcustom w3m-language
   (if (and (boundp 'current-language-environment)
 	   ;; In XEmacs 21.5 it may be the one like "Japanese (UTF-8)".
@@ -5527,8 +5544,8 @@
       (if (and (null cookie)(null body)
 	       (null content-type))
-	   (when w3m-add-user-agent
-	     (list "-header" (concat "User-Agent:" w3m-user-agent)))
+           (list "-header" (concat "User-Agent:"
+             (if w3m-add-user-agent w3m-user-agent "?")))
 	   (when (w3m-add-referer-p url referer)
 	     (list "-header" (concat "Referer: " referer)))
 	   (when w3m-accept-languages
@@ -5536,7 +5553,7 @@
 			      "Accept-Language: "
 			      (mapconcat 'identity w3m-accept-languages
 					 " ")))))
-	(when w3m-add-user-agent (insert "User-Agent: " w3m-user-agent "\n"))
+	(insert "User-Agent: " (if w3m-add-user-agent w3m-user-agent "?") "\n")
 	(when (w3m-add-referer-p url referer)
 	  (insert "Referer: " referer "\n"))
 	(when w3m-accept-languages
@@ -5568,9 +5585,9 @@
   (let ((modes (default-file-modes))
 	(cookie (and w3m-use-cookies (w3m-cookie-get url)))
-    (when w3m-add-user-agent
-      (setq args (nconc args
-			(list "-o" (concat "user_agent=" w3m-user-agent)))))
+    (setq args (nconc args
+      (list "-o" (concat "user_agent="
+        (if w3m-add-user-agent w3m-user-agent "?")))))
     (when cookie
       (setq args (nconc args
 			(list "-header" (concat "Cookie: " cookie)))))
@@ -9996,16 +10013,56 @@
   (w3m-goto-url-new-session "about:blank"))
+(defun w3m-user-agent-change (&optional ua-string)
+"Return a user-agent string.
+Prompt the user to select from entries in
+`w3m-user-agent-default-alist', `w3m-user-agent-alist', or the
+user may manually enter a custom user-agent string.
+When called interactively, variables `w3m-user-agent' and
+`w3m-add-user-agent' are updated, ie. the changes are permanent."
+  (interactive "P")
+  (when (not ua-string)
+    (let ((ua-list (append w3m-user-agent-default-alist w3m-user-agent-alist)))
+     (setq ua-string
+       (or (cdr (assoc (setq ua-string
+                         (completing-read "Select a user-agent: " ua-list))
+                   ua-list))
+           ua-string))))
+  (when (string-equal ua-string "")
+    (setq ua-string nil))
+  (when (called-interactively-p 'interactive)
+    (if (not ua-string)
+      (setq w3m-add-user-agent nil)
+     (setq w3m-add-user-agent t)
+     (setq w3m-user-agent ua-string)))
+  ua-string)
 (defun w3m-reload-this-page (&optional arg no-popup)
   "Reload the current page, disregarding the cached contents.
-If the prefix arg ARG is given, it also clears forms and post data."
+If the prefix arg ARG is given, it also clears forms and post data.
+If the prefix arg is given twice, prompt the user to change the
+user-agent string to be sent for the reload.
+If the prefix arg is given three times, do both, ie. clear forms
+and post data, AND prompt the user to change the user-agent
+string to be sent for the reload."
   (interactive "P")
   (if w3m-current-url
-      (let ((w3m-history-reuse-history-elements
-	     ;; Don't move the history position.
-	     'reload)
-	    post-data)
-	(if arg
+    (let*
+      ((w3m-history-reuse-history-elements 'reload) ;; Don't move the history position.
+       (w3m-user-agent
+         (if (or (equal arg '(16)) (equal arg '(64)))
+           (w3m-user-agent-change)
+          w3m-user-agent))
+       (w3m-add-user-agent
+         (if (or (equal arg '(16)) (equal arg '(64)))
+           w3m-user-agent
+          w3m-add-user-agent))
+       post-data)
+	(if (or (equal arg '(4)) (equal arg '(64)))
 	      (w3m-history-remove-properties '(:forms nil :post-data nil))
 	      (setq w3m-current-forms nil))