Skip to content

/translators

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes U #1332

Merged
merged 9 commits into from Jun 24, 2017
+313 −269

Conversation

@zuphilip
Copy link
Collaborator

commented Jun 17, 2017

I included and expand the fix #1267 by @owcz for USgamer.

owcz and others added some commits Mar 13, 2017

@owcz @zuphilip
date XPath changed
1eb1ab7
@zuphilip
Update USgamer.js 
This and the previous commit include the changes from @owcz
#1267
f2112f9
@zuphilip
Rewrite US National Archives Research Catalog 
The website is using some special loading mechanism
and therfore only hardly scrapable. I wasn't able to
come up with something working stable for multiples.
However, the API works quite fine for single entries
also two nested calls are needed.

Translator request was also formulated here:
https://narations.blogs.archives.gov/2009/08/27/question-if-you-could-personalize-your-experience-of-naras-research-web-pages-and-online-catalog-with-a-customized-user-account-what-features-would-it-include-and-how-would-they-help-you/#comment-113
2dcb1cc
@zuphilip
Update test in UpToDate References.js
6dea033
@zuphilip
Update UPCommons.js
b0c4b4c
@zuphilip
Update tests in unAPI.js
b7a1e30
@zuphilip
Update tests in Ubiquity Journals.js
Loading status checks…
f198b93

@zuphilip zuphilip referenced this pull request Jun 17, 2017

Open

Spanish author names break splitting multiple authors heuristic in EM translators #1333

@adam3smith

adam3smith reviewed Jun 22, 2017
View changes

Copy link
Collaborator

left a comment

Some small requests & questions
UPCommons.js Outdated
translator.setTranslator("951c027d-74ac-47d4-a107-9c3069ab7b48");
translator.setHandler("itemDone", function(obj, item) {
var type = ZU.xpathText(doc, '//meta[@name="DC.type"]/@content');
if (itemTypes[type]!=null) item.itemType = itemTypes[type];

This comment has been minimized.

Sign in to view
Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

simply if (itemType[type]) is all you need here, no?
UPCommons.js
@@ -96,8 +139,13 @@ var testCases = [
"creatorType": "author"
},
{
"firstName": "Miguel Angel",
"lastName": "Ahumada Sempoal",
"firstName": "Ahumada",

This comment has been minimized.

Sign in to view
Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

how confident are you that we can fix the author issue in EM? Seems a bit tricky because we do also find multiple creators in a single DC tag. If we think it might be hard to fix in EM, we'll probably want to parse authors here separately. Your call -- I don't mind taking this while we're working on EM.

This comment has been minimized.

Sign in to view
Copy link
@zuphilip

zuphilip Jun 22, 2017

Author Collaborator

I am not sure if we can , but it would make most sense to try to fix this in EM translator. Therefore I created this issue #1333 to think further afterwards/besides. In the example here we have 9 DC tags each contains only one comma. This could also influence whether to split things up.
US National Archives Research Catalog.js Outdated
if (creators[i].indexOf(", ")>-1) {
item.creators.push(ZU.cleanAuthor(creators[i], "author"));
} else {
item.creators.push({'lastName': creators[i], 'creatorType': 'author'});

This comment has been minimized.

Sign in to view
Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

'fieldMode': true?
US National Archives Research Catalog.js Outdated
"title": "The Struggle for Trade Union Democracy, December 1947",
"creators": [
{
"lastName": "Supreme Commander for the Allied Powers. Economic and Scientific Section. Director for Labor. Labor Division. 8/22/1949-4/28/1952 ",

This comment has been minimized.

Sign in to view
Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

are we sure author fields of that length don't break sync? @dstillman -- what's the max length for creators in sync?

Can

This comment has been minimized.

Sign in to view
Copy link
@zuphilip

zuphilip Jun 24, 2017

Author Collaborator

I tried it out. The maximum length for this field which still will sync is 255. I will check that and shorten if needed.
US National Archives Research Catalog.js Outdated
"title": "Alien Case File for Francisca Torre Vda De Garcia",
"creators": [
{
"lastName": "Department of Justice. Immigration and Naturalization Service. 6/14/1940-3/1/2003 ",

This comment has been minimized.

Sign in to view
Copy link
@adam3smith

adam3smith Jun 22, 2017

Collaborator

looking at these examples, we should at least be able to remove the dates, no?
@zuphilip

This comment has been minimized.

Sign in to view
Copy link
Collaborator Author

commented Jun 24, 2017

Please have a look at the new version.

@adam3smith adam3smith merged commit 4067d6b into zotero:master Jun 24, 2017
1 check passed

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@adam3smith

This comment has been minimized.

Sign in to view
Copy link
Collaborator

commented Jun 24, 2017

Thanks!

@zuphilip zuphilip deleted the zuphilip:fixes-U branch Jun 24, 2017

zuphilip added a commit to zuphilip/translators that referenced this pull request Mar 28, 2018

@zuphilip
Fixes U (zotero#1332) 
* date XPath changed
* Update USgamer.js
This and the previous commit include the changes from @owcz
zotero#1267
* Rewrite US National Archives Research Catalog
The website is using some special loading mechanism
and therefore only hardly scrapeable. I wasn't able to
come up with something working stable for multiples.
However, the API works quite fine for single entries
also two nested calls are needed.

Translator request was also formulated here:
https://narations.blogs.archives.gov/2009/08/27/question-if-you-could-personalize-your-experience-of-naras-research-web-pages-and-online-catalog-with-a-customized-user-account-what-features-would-it-include-and-how-would-they-help-you/#comment-113

* Update test in UpToDate References.js
* Update UPCommons.js
* Update tests in unAPI.js
* Update tests in Ubiquity Journals.js
0e96e47

zuphilip added a commit to zuphilip/translators that referenced this pull request Mar 28, 2018

@zuphilip
Fixes U (zotero#1332) 
* date XPath changed
* Update USgamer.js
This and the previous commit include the changes from @owcz
zotero#1267
* Rewrite US National Archives Research Catalog
The website is using some special loading mechanism
and therefore only hardly scrapeable. I wasn't able to
come up with something working stable for multiples.
However, the API works quite fine for single entries
also two nested calls are needed.

Translator request was also formulated here:
https://narations.blogs.archives.gov/2009/08/27/question-if-you-could-personalize-your-experience-of-naras-research-web-pages-and-online-catalog-with-a-customized-user-account-what-features-would-it-include-and-how-would-they-help-you/#comment-113

* Update test in UpToDate References.js
* Update UPCommons.js
* Update tests in unAPI.js
* Update tests in Ubiquity Journals.js
d7ee399
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adam3smith adam3smith

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@zuphilip @adam3smith @owcz
You can’t perform that action at this time.