Re: sb-atom and sb-rss fetching already deleted articles

[Katsumi Yamaoka <yamaoka@xxxxxxx>]

>>>>> In [emacs-w3m : No.11138] David Engster wrote:
> +2010-02-17  David Engster  <dengste@xxxxxx>
> +
> +	* sb-atom.el (shimbun-atom-get-headers): When encountering an already
> +	present article, stop fetching further entries.
> +	* sb-rss.el (shimbun-rss-get-headers): Same change as in sb-atom.

I realized that the change in sb-rss.el blocks sb-itmedia.el from
fetching new articles at least for the news.bursts group[1].  It
is because there is sometimes an old article that keeps remaining
at the beginning of the feed.  It makes sb-itmedia.el immediately
quit fetching.  Maybe it can be solved by setting the 4th argument
passed to 'shimbun-rss-get-headers' as follows:

--- sb-itmedia.el~	2010-02-10 04:48:01 +0000
+++ sb-itmedia.el	2010-02-24 10:00:18 +0000
@@ -121,7 +121,7 @@
 (luna-define-method shimbun-get-headers :around ((shimbun shimbun-itmedia)
 						 &optional range)
   (if (string-match "\\.xml\\'" (shimbun-index-url shimbun))
-      (luna-call-next-method)
+      (shimbun-rss-get-headers shimbun range t t)
     (let ((case-fold-search t)
 	  (group (nth 2 (assoc (shimbun-current-group-internal shimbun)
 			       shimbun-itmedia-group-alist)))

[1] http://rss.itmedia.co.jp/rss/2.0/news_bursts.xml

However, the sb-rss.el change should affect not only sb-itmedia.el
but also the other shimbun modules that use sb-rss.el.  It means we
should add the wrapper like the following to the rss-based modules
except for some exceptions.

(luna-define-method shimbun-get-headers ((shimbun shimbun-foobar)
					 &optional range)
  (shimbun-rss-get-headers shimbun range t t))

But it will be very troublesome.  Instead, how about not changing
the default behavior of `shimbun-rss-get-headers'?  Even in this
case, it is made immediately quit when the header that has ever
been fetched is found, if the 5th argument, that is newly added,
is set.  And only the modules that want it to work so need to have
the following wrapper:

(luna-define-method shimbun-get-headers ((shimbun shimbun-foobar)
					 &optional range)
  (shimbun-rss-get-headers shimbun range t nil t))

The new `shimbun-rss-get-headers' will be as follows:

--- sb-rss.el~	2010-02-18 22:17:43 +0000
+++ sb-rss.el	2010-02-24 10:00:18 +0000
@@ -173,7 +173,8 @@
   (shimbun-rss-get-headers shimbun range t))
 
 (defun shimbun-rss-get-headers (shimbun &optional range
-					need-descriptions need-all-items)
+					need-descriptions need-all-items
+					quit-immediately)
   (let ((xml (condition-case err
 		 (xml-parse-region (point-min) (point-max))
 	       (error
@@ -216,10 +217,12 @@
 			 (id (shimbun-rss-build-message-id shimbun url date))
 			 (subject (shimbun-rss-node-text rss-ns 'title item)))
 		    (when (and id
-			       (shimbun-search-id shimbun id)
-			       (not need-all-items))
-		      (throw 'done headers))
-		    (when (and id
+			       (or need-all-items
+				   (if (shimbun-search-id shimbun id)
+				       (if quit-immediately
+					   (throw 'done headers)
+					 nil)
+				     t))
 			       (if (and ignored-subject subject)
 				   (not (string-match ignored-subject subject))
 				 t))