[Netarchivesuite-users] About the crawler traps
Soleto Ruiz de Clavijo, Miguel
miguel.soleto at externos.bne.es
Tue Aug 12 09:18:01 CEST 2025
Dear all,
I have a question about traps. We have identified thousands of 404 codes in our crawls and want to add them as traps in the harvest. However, there are over 23,000 of them, and when I try to save them, I get a 502 error.
Is there any way to add all these traps?
Thank you very much in advance for your help.
Best regards,
Miguel.
________________________________
Este mensaje y cualquier fichero adjunto est?n dirigidos ?nicamente a sus destinatarios y contiene informaci?n confidencial. Si usted ha recibido este correo electr?nico por error, le informamos que no puede realizar ninguna revisi?n, alteraci?n, impresi?n, copia, transmisi?n, difusi?n ni utilizaci?n alguna de este mensaje ni de cualquier fichero adjunto que pudiese contener. La realizaci?n de cualquiera de los actos indicados est? expresamente prohibida por las Normas que regulan estas materias. Por todo ello se solicita que, en caso de existir error en la recepci?n de este mensaje, se lo notifique al remitente respondiendo a este e-mail y elimine el mensaje y su contenido inmediatamente. La Biblioteca Nacional de Espa?a se reserva las acciones legales que le correspondan en el caso de que se infrinja lo indicado anteriormente.
________________________________
The information in this e-mail and any attachments is confidential and it is intended for the addressee only. If you have received this e-mail in error, you are notified that any revision, amendment, print, copy, disclosure, distribution or use of the contents is unauthorized. Carrying out any of the above actions, is expressly banned by rules governing this matter. Hence we request that if you are not the intended recipient, please notify the sender answering this e-mail, and delete the message and any attachments. The National Library of Spain reserves itself the right to take the appropriate legal actions in the event of the above mentioned matter is being infringed.
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20250812/26f0575b/attachment.html>
More information about the NetarchiveSuite-users
mailing list