[Netarchivesuite-users] Draft of NAS template for Heritrix3

Søren Vejrup Carlsen svc at kb.dk
Tue Mar 17 12:33:36 CET 2015


Hi all.

The attached file is a draft of how the NAS template for Heritrix3 will look like (Also attached is the H1 template at the basis of the new H3 template).  The overrides were generated by the  MigrateH1toH3Tool.
Embedded in this file is placeholders marked with %{..}, most of which will be mandatory.
These placeholders are substituted  at job-generation time. There are currently the following placeholders:

%{MAX_TIME_SECONDS_PLACEHOLDER}
%{FRONTIER_QUEUE_TOTAL_BUDGET_PLACEHOLDER}
%{CRAWLERTRAPS_PLACEHOLDER}
%{ARCHIVER_PROCESSOR_BEAN_PLACEHOLDER}
%{ARCHIVER_BEAN_REFERENCE_PLACEHOLDER}
%{QUOTA_ENFORCER_GROUP_MAX_FETCH_SUCCES_PLACEHOLDER}
%{QUOTA_ENFORCER_MAX_BYTES_PLACEHOLDER}

At the beginning of the template is a lot of overrides for the default properties in the components of the Heritrix3 components.
I believe, that this is a good idea to have the overrides here instead of all over the template.

Outstanding is the addition of the DeDuplicator placeholder(s).

Best Regards
---------------------------------------------------------------------------
Søren Vejrup Carlsen, Department of Digital Preservation, Royal Library, Copenhagen, Denmark
tlf: (+45) 33 47 48 41
email: svc at kb.dk
----------------------------------------------------------------------------
Non omnia possumus omnes
--- Macrobius, Saturnalia, VI, 1, 35 -------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20150317/def7b7ea/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: default_orderxml_h3_nodedup_new.xml
Type: text/xml
Size: 21980 bytes
Desc: default_orderxml_h3_nodedup_new.xml
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20150317/def7b7ea/attachment-0001.xml>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: H1_template_origin.xml.txt
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20150317/def7b7ea/attachment-0001.txt>


More information about the NetarchiveSuite-users mailing list