Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes U #1332

Merged
merged 9 commits into from Jun 24, 2017

Conversation

Projects
None yet
3 participants
@zuphilip
Copy link
Collaborator

zuphilip commented Jun 17, 2017

I included and expand the fix #1267 by @owcz for USgamer.

owcz and others added some commits Mar 13, 2017

Update USgamer.js
This and the previous commit include the changes from @owcz
#1267
Rewrite US National Archives Research Catalog
The website is using some special loading mechanism
and therfore only hardly scrapable. I wasn't able to
come up with something working stable for multiples.
However, the API works quite fine for single entries
also two nested calls are needed.

Translator request was also formulated here:
https://narations.blogs.archives.gov/2009/08/27/question-if-you-could-personalize-your-experience-of-naras-research-web-pages-and-online-catalog-with-a-customized-user-account-what-features-would-it-include-and-how-would-they-help-you/#comment-113
@adam3smith
Copy link
Collaborator

adam3smith left a comment

Some small requests & questions

translator.setTranslator("951c027d-74ac-47d4-a107-9c3069ab7b48");
translator.setHandler("itemDone", function(obj, item) {
var type = ZU.xpathText(doc, '//meta[@name="DC.type"]/@content');
if (itemTypes[type]!=null) item.itemType = itemTypes[type];

This comment has been minimized.

Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

simply if (itemType[type]) is all you need here, no?

@@ -96,8 +139,13 @@ var testCases = [
"creatorType": "author"
},
{
"firstName": "Miguel Angel",
"lastName": "Ahumada Sempoal",
"firstName": "Ahumada",

This comment has been minimized.

Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

how confident are you that we can fix the author issue in EM? Seems a bit tricky because we do also find multiple creators in a single DC tag. If we think it might be hard to fix in EM, we'll probably want to parse authors here separately. Your call -- I don't mind taking this while we're working on EM.

This comment has been minimized.

Copy link
@zuphilip

zuphilip Jun 22, 2017

Author Collaborator

I am not sure if we can , but it would make most sense to try to fix this in EM translator. Therefore I created this issue #1333 to think further afterwards/besides. In the example here we have 9 DC tags each contains only one comma. This could also influence whether to split things up.

if (creators[i].indexOf(", ")>-1) {
item.creators.push(ZU.cleanAuthor(creators[i], "author"));
} else {
item.creators.push({'lastName': creators[i], 'creatorType': 'author'});

This comment has been minimized.

Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

'fieldMode': true?

"title": "The Struggle for Trade Union Democracy, December 1947",
"creators": [
{
"lastName": "Supreme Commander for the Allied Powers. Economic and Scientific Section. Director for Labor. Labor Division. 8/22/1949-4/28/1952 ",

This comment has been minimized.

Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

are we sure author fields of that length don't break sync? @dstillman -- what's the max length for creators in sync?

Can

This comment has been minimized.

Copy link
@zuphilip

zuphilip Jun 24, 2017

Author Collaborator

I tried it out. The maximum length for this field which still will sync is 255. I will check that and shorten if needed.

"title": "Alien Case File for Francisca Torre Vda De Garcia",
"creators": [
{
"lastName": "Department of Justice. Immigration and Naturalization Service. 6/14/1940-3/1/2003 ",

This comment has been minimized.

Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

looking at these examples, we should at least be able to remove the dates, no?

@zuphilip

This comment has been minimized.

Copy link
Collaborator Author

zuphilip commented Jun 24, 2017

Please have a look at the new version.

@adam3smith adam3smith merged commit 4067d6b into zotero:master Jun 24, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@adam3smith

This comment has been minimized.

Copy link
Collaborator

adam3smith commented Jun 24, 2017

Thanks!

@zuphilip zuphilip deleted the zuphilip:fixes-U branch Jun 24, 2017

zuphilip added a commit to zuphilip/translators that referenced this pull request Mar 28, 2018

Fixes U (zotero#1332)
* date XPath changed
* Update USgamer.js
This and the previous commit include the changes from @owcz
zotero#1267
* Rewrite US National Archives Research Catalog
The website is using some special loading mechanism
and therefore only hardly scrapeable. I wasn't able to
come up with something working stable for multiples.
However, the API works quite fine for single entries
also two nested calls are needed.

Translator request was also formulated here:
https://narations.blogs.archives.gov/2009/08/27/question-if-you-could-personalize-your-experience-of-naras-research-web-pages-and-online-catalog-with-a-customized-user-account-what-features-would-it-include-and-how-would-they-help-you/#comment-113

* Update test in UpToDate References.js
* Update UPCommons.js
* Update tests in unAPI.js
* Update tests in Ubiquity Journals.js

zuphilip added a commit to zuphilip/translators that referenced this pull request Mar 28, 2018

Fixes U (zotero#1332)
* date XPath changed
* Update USgamer.js
This and the previous commit include the changes from @owcz
zotero#1267
* Rewrite US National Archives Research Catalog
The website is using some special loading mechanism
and therefore only hardly scrapeable. I wasn't able to
come up with something working stable for multiples.
However, the API works quite fine for single entries
also two nested calls are needed.

Translator request was also formulated here:
https://narations.blogs.archives.gov/2009/08/27/question-if-you-could-personalize-your-experience-of-naras-research-web-pages-and-online-catalog-with-a-customized-user-account-what-features-would-it-include-and-how-would-they-help-you/#comment-113

* Update test in UpToDate References.js
* Update UPCommons.js
* Update tests in unAPI.js
* Update tests in Ubiquity Journals.js
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.