I'm using this last post to link to a couple of the documents I've created, all of which are living on the wiki.
The first is a Web Site Collection Plan. This draft suggests policies related to archiving web sites within UARMS.
The other is a Web Site Archiving Workflow and General Information document. This outlines a suggested general workflow, instructions on using HTTrack and the NetarchiveSuite Quick Start version, and information related to the crawls I've done toward the end of this semester.
I have no plans to take down the wiki or this blog anytime in the near future, so these documents should be available until further notice.
Tuesday, April 28, 2009
Thursday, April 23, 2009
April 23, 2009
My fellowship practicum ends next Monday, so this may be my last post.
The best news, perhaps of the semester, is that my ibiblio.org contact got Web Curator Tool (WCT) up and running for me. I've been testing different parts of the program, and there are some bugs (I haven't been able to actually run a harvest) but I've gotten a feel for the procedure the software imposes (harvest authorisation -> target creation -> harvest approval -> target instance -> harvest). The interface is definitely more approachable compared with NetarchiveSuite and HTTrack, down to the colorful grains of wheat next to the "Harvester Configuration" box.
It can still feel a little complex, clicking back and forth between harvest authorisation and target, but I think this is due to (1) my lack of experience with the program (2) some lingering bugs and (3) the fact that setting up the authorisations and targets would require more time investment in the beginning than after your staple harvests were configured.
One point about WCT that vexes me is the difficult install. If I was presented with these three tools, without any prior experience, and told to evaluate them based on interface alone, I'd choose WCT above the others in a heartbeat. But ease-of-use balanced against installation tribulations is a harder decision to make. Not being the installer doesn't really alleviate that tension for me, either.
Also, the harvest authorisation feature of WCT would not be an especially useful aspect of the program for archiving UNC sites. This can be done without gaining the creator's permission based on public records law. The HA feature then becomes a superfluous extra step. On the other side of that coin, in the future, if UARMS or another department wishes to expand the scope of their web site archive to other sites for which permission is needed, this feature would become very useful, because it helps keep the temptation to crawl on the sly at bay.
Another feature of WCT that could work to slow or facilitate the workflow is the harvest approval. If you're a lone harvester, getting approval from yourself is just an extra step. But if multiple people are wrangling and suggesting different harvests, this helps portion out the resources.
I can't comment on the harvesting feature, but will do so if we can make it work in the next few days. Testing it has held up the opinion I gave at the end of my poster: right now, HTTrack is a good option for UARMS, but WCT would be a very nice alternative in the (technologically-supported and web-site-archiving-program-developed) future.
The best news, perhaps of the semester, is that my ibiblio.org contact got Web Curator Tool (WCT) up and running for me. I've been testing different parts of the program, and there are some bugs (I haven't been able to actually run a harvest) but I've gotten a feel for the procedure the software imposes (harvest authorisation -> target creation -> harvest approval -> target instance -> harvest). The interface is definitely more approachable compared with NetarchiveSuite and HTTrack, down to the colorful grains of wheat next to the "Harvester Configuration" box.
It can still feel a little complex, clicking back and forth between harvest authorisation and target, but I think this is due to (1) my lack of experience with the program (2) some lingering bugs and (3) the fact that setting up the authorisations and targets would require more time investment in the beginning than after your staple harvests were configured.One point about WCT that vexes me is the difficult install. If I was presented with these three tools, without any prior experience, and told to evaluate them based on interface alone, I'd choose WCT above the others in a heartbeat. But ease-of-use balanced against installation tribulations is a harder decision to make. Not being the installer doesn't really alleviate that tension for me, either.
Also, the harvest authorisation feature of WCT would not be an especially useful aspect of the program for archiving UNC sites. This can be done without gaining the creator's permission based on public records law. The HA feature then becomes a superfluous extra step. On the other side of that coin, in the future, if UARMS or another department wishes to expand the scope of their web site archive to other sites for which permission is needed, this feature would become very useful, because it helps keep the temptation to crawl on the sly at bay.
Another feature of WCT that could work to slow or facilitate the workflow is the harvest approval. If you're a lone harvester, getting approval from yourself is just an extra step. But if multiple people are wrangling and suggesting different harvests, this helps portion out the resources.
I can't comment on the harvesting feature, but will do so if we can make it work in the next few days. Testing it has held up the opinion I gave at the end of my poster: right now, HTTrack is a good option for UARMS, but WCT would be a very nice alternative in the (technologically-supported and web-site-archiving-program-developed) future.
Monday, April 6, 2009
April 6, 2009
I spent last week volunteering for and attending DigCCurr2009 here in Chapel Hill, which was robustly attended by almost 300 people from around the world. It was an informative experience, and as always a nice chance to renew my enthusiasm by seeing that there are others out there working on similar problems and coming up with different paths forward.
On Thursday, I presented my poster. A lot of good questions came my way. As I've mentioned before, this poster describes my work here in UARMS evaluating three open source software tools used for archiving websites. Those tools are HTTrack, NetarchiveSuite, and Web Curator Tool (WCT).
All three tools would be good choices, and scored very high. HTTrack came out on top with a score of 29. WCT was close behind with 28 points. NetarchiveSuite was third, but still had 25 points.
My recommendation, right now, is for UARMS to go with HTTrack (with WCT as runner-up). This was surprising to me, since I had originally considered it to be the least enticing option. It definitely has the fewest features (no metadata assignment, preservation functions, permissions tracking), but it is the simplest to install and run. I suppose, the moral of this story is that your situation and personnel dictate your best software choice. The thoroughbred may not always be the right horse for you. HTTrack will be easiest for any user to pick up and operate without extensive time reading documentation or learning about archiving websites.
On Thursday, I presented my poster. A lot of good questions came my way. As I've mentioned before, this poster describes my work here in UARMS evaluating three open source software tools used for archiving websites. Those tools are HTTrack, NetarchiveSuite, and Web Curator Tool (WCT).
All three tools would be good choices, and scored very high. HTTrack came out on top with a score of 29. WCT was close behind with 28 points. NetarchiveSuite was third, but still had 25 points.
My recommendation, right now, is for UARMS to go with HTTrack (with WCT as runner-up). This was surprising to me, since I had originally considered it to be the least enticing option. It definitely has the fewest features (no metadata assignment, preservation functions, permissions tracking), but it is the simplest to install and run. I suppose, the moral of this story is that your situation and personnel dictate your best software choice. The thoroughbred may not always be the right horse for you. HTTrack will be easiest for any user to pick up and operate without extensive time reading documentation or learning about archiving websites.
Subscribe to:
Posts (Atom)