Skip to content
Please note that GitHub no longer supports your web browser.

We recommend upgrading to the latest Google Chrome or Firefox.

Learn more
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New translator for Cowles Foundation publications #2061

Open
wants to merge 21 commits into
base: master
from

Conversation

@placardo
Copy link
Contributor

placardo commented Nov 21, 2019

A translator for the documents on the Cowles Foundation website (https://cowles.yale.edu/).
The translator works on the authors' pages and on the different lists of publications (for monographs, discussion papers and published articles), and gets pdfs when available.
I tried to follow the guidelines at https://www.zotero.org/support/dev/translators.

Copy link
Collaborator

adam3smith left a comment

Thanks!
Some suggestions to clean up the code a bit & make it more robust

Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
Cowles.js Outdated Show resolved Hide resolved
placardo added 3 commits Nov 22, 2019
@placardo

This comment has been minimized.

Copy link
Contributor Author

placardo commented Nov 22, 2019

Thanks!
Some suggestions to clean up the code a bit & make it more robust

Thank you for your quick feedback! This is my first translator and I'm not used to Github so I apologize in advance for any silly mistakes

@zuphilip zuphilip requested a review from adam3smith Nov 25, 2019
@adam3smith

This comment has been minimized.

Copy link
Collaborator

adam3smith commented Dec 5, 2019

This all looks very good now thanks (and apologies for the delay).
Two remaining issues:

  1. The root that you specify -- that shouldn't be necessary. Zotero can handle relative links and they're more robust. Could you test if all of this still works entirely without root and then remove it? (If you're used to other webscraping tools e.g. in R or python, they tend to behave less like a browser and break with relative links; Zotero doesn't)

  2. Please use the minimized version of text() and attr() as e.g. here: https://github.com/zotero/translators/blob/master/Emerald%20Insight.js#L37 use all three lines -- the first one identifies, the 2nd disables linting, the 3rd is the minimized code.

@placardo

This comment has been minimized.

Copy link
Contributor Author

placardo commented Dec 5, 2019

I am not sure how to do without the root part because, if I understand it correctly, Zotero inserts the current url argument if I don't specify it; while it would work on the items pages or the list of publications, it would break eg on an authors' page where the current url does not match the beginning of an item's url (for example: you're on this page https://cowles.yale.edu/author/herbert-e-scarf, and select an item to import to zotero. The current url used is https://cowles.yale.edu/author and if I just append the item url for "/cfdp-2212" it throws a 404 because the actual beginning should be https://cowles.yale.edu/publications/cfdp). Same goes for the PDF, as what is scraped is an address starting at the root of the site; if I delete var root it sends a GET to the current URL (eg. https://cowles.yale.edu/publications/archives/cfm) + the scraped address whereas it should only send a GET to https://cowles.yale.edu/ + the scraped address...

Maybe I missed something because I did not take the time yet to see how Zotero connector works please tell me if you think I could do it using only relative links

@adam3smith

This comment has been minimized.

Copy link
Collaborator

adam3smith commented Dec 5, 2019

Did you test? Relative links always refer to the host, i.e what you call the root and that's what Zotero uses (or certainly should use). In other words root + "/" + variable is essentially identical to "/" + variable but will behave better e.g. if URL proxies are involved or with http/https differences

@placardo

This comment has been minimized.

Copy link
Contributor Author

placardo commented Dec 5, 2019

Sorry I was confused it seems I got that error by testing without the initial /, in that case apparently Zotero appends the entire current URL and not just the host. It seems to work now with the initial /

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.