<font size=2 face="sans-serif">Hello Alicia,</font><br><br><font size=2 face="sans-serif">You have to check which JobGenerator
system you are using in NAS settings.</font><br><font size=2 face="sans-serif">At BnF, we use the FixedDomainConfigurationCountJobGenerator
which gives us the possibility to </font><br><font size=2 face="sans-serif">put a defined number of domains per
job in snapshot harvests and another defined number of domains per job
in focused crawls.</font><br><br><font size=2 face="sans-serif">Best,</font><br><br><font size=2 face="sans-serif">Sara</font><br><br><font size=2 face="sans-serif">
<scheduler></font><br><font size=2 face="sans-serif">
<limitSubmittedJobsInQueue>true</limitSubmittedJobsInQueue></font><br><font size=2 face="sans-serif">
<submittedJobsInQueueLimit>1</submittedJobsInQueueLimit></font><br><font size=2 face="sans-serif">
<jobtimeouttime>31536000</jobtimeouttime></font><br><font size=2 face="sans-serif">
<jobgenerationperiod>60</jobgenerationperiod></font><br><font size=2 face="sans-serif">
<jobGen></font><br><font size=2 face="sans-serif">
<class><b>dk.netarkivet.harvester.scheduler.jobgen.FixedDomainConfigurationCountJobGenerator</b></class></font><br><font size=2 face="sans-serif">
<objectLimitIsSetByQuotaEnforcer>false</objectLimitIsSetByQuotaEnforcer></font><br><font size=2 face="sans-serif">
<domainConfigSubsetSize>5000</domainConfigSubsetSize></font><br><font size=2 face="sans-serif">
<config></font><br><font size=2 face="sans-serif">
<b> <fixedDomainCountSnapshot>5000</fixedDomainCountSnapshot></b></font><br><font size=2 face="sans-serif"><b>
<fixedDomainCountFocused>500</fixedDomainCountFocused></b></font><br><font size=2 face="sans-serif">
<excludeDomainsWithZeroBudget>true</excludeDomainsWithZeroBudget></font><br><font size=2 face="sans-serif">
<postponeUnregisteredChannel>false</postponeUnregisteredChannel></font><br><font size=2 face="sans-serif">
</config></font><br><font size=2 face="sans-serif">
</jobGen></font><br><font size=2 face="sans-serif">
</scheduler></font><br><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">De :
</font><font size=1 face="sans-serif">"Pastrana García,
Alicia" <alicia.pastrana@bne.es></font><br><font size=1 color=#5f5f5f face="sans-serif">A :
</font><font size=1 face="sans-serif">"geraldine.camile@bnf.fr"
<geraldine.camile@bnf.fr>, "netarchivesuite-curator@ml.sbforge.org"
<netarchivesuite-curator@ml.sbforge.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Cc :
</font><font size=1 face="sans-serif">"bert.wendland@bnf.fr"
<bert.wendland@bnf.fr>, "clara.wiatrowski@bnf.fr" <clara.wiatrowski@bnf.fr>,
"DDL_DLN@bnf.fr" <DDL_DLN@bnf.fr>, "leslie.bellony-ext@bnf.fr"
<leslie.bellony-ext@bnf.fr></font><br><font size=1 color=#5f5f5f face="sans-serif">Date :
</font><font size=1 face="sans-serif">02/07/2019 13:33</font><br><font size=1 color=#5f5f5f face="sans-serif">Objet :
</font><font size=1 face="sans-serif">Re: [Netarchivesuite-curator]
BnF NAS update for July</font><br><font size=1 color=#5f5f5f face="sans-serif">Envoyé par :
</font><font size=1 face="sans-serif">"Netarchivesuite-curator"
<netarchivesuite-curator-bounces@ml.sbforge.org></font><br><hr noshade><br><br><br><font size=3 color=#004080 face="Calibri">Hello all,</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">Here is our update:</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">We are working in our three
event crawl: European Parliament elections, local elections and Spanish
Government elections that are still running. We have had a great collaboration
from the different regions in the local elections, and we nominate over
3.700 sites.</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">We still don’t know when
we are going to launch our broad crawl but it will probably be in September</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">This is the problem that
we have that I have told you. I hope you can understand:</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">There is a huge harvest in
one of our collections and we can’t crawl all the seeds in it. We have
a problem with the division in Jobs of the harvest. For example, with version
5.3 we had loaded 15,000 URLs in a harvest and it generated 29 jobs of
620 URLs and one job with the rest .. When update to 5.4, it generate Jobs
of 2096 URLs, which creates a local disk problem in the spiders because
it is small. We use the same template as in 5.3 but we don’t know why
the division is different. Do you know what this can be? Is there a parameter
in NAS (or templates) that we can modify to reduce the number of URLs generated
in each job?</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">If you have any questions
about this, please do not hesitate to ask me.</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#004080 face="Calibri">Thank you!</font><br><font size=3 color=#004080 face="Calibri"> </font><br><font size=3 color=#808080 face="sans-serif">Alicia Pastrana García</font><br><font size=3 color=#808080 face="sans-serif">Área de Gestión del Depósito
de las Publicaciones en Línea</font><br><font size=3 color=#808080 face="sans-serif">División de Procesos y
Servicios Digitales</font><br><font size=3 color=#808080 face="sans-serif">Tfno.: 91 516 89 92</font><br><font size=3 color=#808080 face="sans-serif">Biblioteca Nacional de
España</font><br><font size=3 face="Times New Roman"> </font><br><font size=3 face="Tahoma"><b>De:</b> Netarchivesuite-curator [</font><a href="mailto:netarchivesuite-curator-bounces@ml.sbforge.org"><font size=3 face="Tahoma">mailto:netarchivesuite-curator-bounces@ml.sbforge.org</font></a><font size=3 face="Tahoma">]
<b>En nombre de </b>geraldine.camile@bnf.fr<b><br>Enviado el:</b> martes, 02 de julio de 2019 11:50<b><br>Para:</b> netarchivesuite-curator@ml.sbforge.org<b><br>CC:</b> bert.wendland@bnf.fr; leslie.bellony-ext@bnf.fr; DDL_DLN@bnf.fr;
clara.wiatrowski@bnf.fr<b><br>Asunto:</b> [Netarchivesuite-curator] BnF NAS update for July</font><br><font size=3 face="Times New Roman"> </font><br><font size=3 face="Arial">Hello all,</font><font size=3 face="Times New Roman"><br></font><font size=3 face="Arial"><br>In March, we launched a selective project crawl for the European elections
which is to come to an end in the coming days. 15 curators contributed
to the nomination of over 1480 sites among which social networks (twitter
mostly but also facebook and Youtube channels) represent the largest share
(around 60%). Eventually, 18 weekly, 5 monthly and over 120 daily crawls
were led. We contributed for 85 sites to the collaborative crawl launched
by Ricardo Basilio on European elections results.</font><font size=3 face="Times New Roman"><br></font><font size=3 face="Arial"><br>We also added our contribution to the collaborative crawl on Artificial
intelligence (85 sites).</font><font size=3 face="Times New Roman"><br></font><font size=3 face="Arial"><br>Best regards,<br>The BnF digital legal deposit team</font><div align=center><hr></div><p><font size=3 face="Arial">Expositions </font><a href="https://www.bnf.fr/fr/agenda/manuscrits-de-lextreme"><font size=3 color=blue face="Arial"><b><i><u>Manuscrits
de l’extrême </u></i></b></font></a><font size=3 face="Arial">– jusqu'au
7 juillet 2019 | François-Mitterrand<br>et </font><a href="https://www.bnf.fr/fr/agenda/le-monde-en-spheres"><font size=3 color=blue face="Arial"><b><i><u>Le
Monde en sphères</u></i></b></font></a><font size=3 face="Arial"> – jusqu'au
21 juillet 2019 | François-Mitterrand</font><p><font size=3 color=#008000 face="Arial"><b>Avant d'imprimer, pensez
à l'environnement.</b></font><p><hr><font size=1>Este mensaje y cualquier fichero adjunto están dirigidos
únicamente a sus destinatarios y contiene información confidencial. Si
usted ha recibido este correo electrónico por error, le informamos que
no puede realizar ninguna revisión, alteración, impresión, copia, transmisión,
difusión ni utilización alguna de este mensaje ni de cualquier fichero
adjunto que pudiese contener. La realización de cualquiera de los actos
indicados está expresamente prohibida por las Normas que regulan estas
materias. Por todo ello se solicita que, en caso de existir error en la
recepción de este mensaje, se lo notifique al remitente respondiendo a
este e-mail y elimine el mensaje y su contenido inmediatamente. La Biblioteca
Nacional de España se reserva las acciones legales que le correspondan
en el caso de que se infrinja lo indicado anteriormente.</font><font size=3></font><hr><font size=1>The information in this e-mail and any attachments is
confidential and it is intended for the addressee only. If you have received
this e-mail in error, you are notified that any revision, amendment, print,
copy, disclosure, distribution or use of the contents is unauthorized.
Carrying out any of the above actions, is expressly banned by rules governing
this matter. Hence we request that if you are not the intended recipient,
please notify the sender answering this e-mail, and delete the message
and any attachments. The National Library of Spain reserves itself the
right to take the appropriate legal actions in the event of the above mentioned
matter is being infringed.</font><font size=3> </font><hr><tt><font size=2>_______________________________________________<br>Netarchivesuite-curator mailing list<br>Netarchivesuite-curator@ml.sbforge.org<br></font></tt><a href="https://ml.sbforge.org/mailman/listinfo/netarchivesuite-curator"><tt><font size=2>https://ml.sbforge.org/mailman/listinfo/netarchivesuite-curator</font></tt></a><tt><font size=2><br></font></tt><p><font face="sans-serif"><hr />
<p>Expositions <strong><em><a href="https://www.bnf.fr/fr/agenda/manuscrits-de-lextreme">Manuscrits de l’extrême </a></em></strong> – jusqu'au 7 juillet 2019 | François-Mitterrand<br />et <strong><em><a href="https://www.bnf.fr/fr/agenda/le-monde-en-spheres">Le Monde en sphères</a></em></strong> – jusqu'au 21 juillet 2019 | François-Mitterrand</p>
<p style="color:#008000"><strong>Avant d'imprimer, pensez à l'environnement.</strong></p></font>