Make ArchiveRecord.getContentBytes consistent, Resolve #334 #335

ianmilligan1 · Jul 30, 2019

GitHub issue(s):

What does this Pull Request do?

As noted in #334, @jrwiebe discovered that we are inconsistent on how we getContentBytes of ARC and WARC files. Currently, ARC files we just get the contents of the record (using getBodyContent) and WARC files we get everything (using getContent). The two should be consistent.

On the ticket itself, I discussed the various outputs, how they differ, and potential solutions. I think it is critical that we normalize the two approaches. Given the existence of RemoveHttpHeader, I think we should have both just using getContent. If people just want the body content, they can remove the header (which is what we've been doing with WARC files). I can imagine lots of diverse use cases for HTTP header information so it's better to have it in there and then remove it.

How should this be tested?

TravisCI should turn green. Note that I had to change the tests because now that we are returning headers in the ARC files, the tika and language counts are different. This is to be expected, to me at least.
We should do some sanity checking on the outputs.

Additional Notes:

We should probably foreground RemoveHttpHeader more in our documentation.

Interested parties

@ruebot @jrwiebe

codecov-io · Jul 30, 2019

Codecov Report

Merging #335 into master will decrease coverage by 0.97%.
The diff coverage is 100%.

@@           Coverage Diff            @@
##           master   #335      +/-   ##
========================================
- Coverage   75.97%    75%   -0.98%     
========================================
  Files          39     39              
  Lines        1124   1124              
  Branches      197    197              
========================================
- Hits          854    843      -11     
- Misses        205    214       +9     
- Partials       65     67       +2

Impacted Files	Coverage Δ
...ain/scala/io/archivesunleashed/ArchiveRecord.scala	`84.9% <100%> (ø)`	⬆️
...io/archivesunleashed/matchbox/DetectLanguage.scala	`66.66% <0%> (-33.34%)`	⬇️
...rchivesunleashed/matchbox/DetectMimeTypeTika.scala	`75% <0%> (-25%)`	⬇️
...java/io/archivesunleashed/data/ArcRecordUtils.java	`63.04% <0%> (-15.22%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d14b92...5c1637b. Read the comment docs.

ruebot · Aug 1, 2019

@ianmilligan1 I'm going to test with language and full text extraction. That the big stuff? Also, do you want to create a ticket for documentation so it doesn't get lost in the shuffle?

ruebot · Aug 1, 2019

Plain text

Script

import io.archivesunleashed._
import io.archivesunleashed.matchbox._

RecordLoader.loadArchives("/home/nruest/Projects/au/aut-resources/Sample-Data/*.gz", sc)
  .keepValidPages()
  .map(r => (r.getCrawlDate, r.getDomain, r.getUrl, RemoveHTML(r.getContentString)))
  .saveAsTextFile("/tmp/plain-text/")

sys.exit

Results

$ ls /tmp/plain-text/
part-00000  part-00001  _SUCCESS

$ head -n 2 /tmp/plain-text/part-00000
(20091218,www.equalvoice.ca,http://www.equalvoice.ca/images/images/french/js/images/sponsors/enbridge.jpg,HTTP/1.1 200 OK Connection: close Date: Fri, 18 Dec 2009 23:17:29 GMT Server: Microsoft-IIS/6.0 PICS-Label: (PICS-1.0 "http://www.rsac.org/ratingsv01.html" l by "info@xynapse.ca" on "2004.10.12T16:51-0400" exp "2010.10.12T12:00-0400" r (v 0 s 0 n 0 l 0)) X-Powered-By: ASP.NET Content-Language: en-CA Content-Type: text/html; charset=UTF-8 Equal Voice  Equal Voice HOME | ENGLISH | FRENCH RSS SUBSCRIBE About Us Mission Advisory Board Board of Directors & Staff Programs and Events EV Programs EV Events EV Speaks Out Research and Reports Reports Fundamental Facts Women in Canadian Politics Chapters NCR Ottawa British Columbia Alberta North Alberta South Youth Founders (Toronto) Toronto Municipal Nova Scotia Newfoundland & Labrador Join and Act Become an EV member Take Action Media News Releases In The News Video Photo Gallery Presentations Sponsors Donate Contact Us Contact Us Job Postings EV SPEAKS OUT Latest statements from Equal Voice Nov 19, 2009 Sheila James Fund to be established at Equal Voice Nov 11, 2009 Bulletin d’AVE - automne 2009 MORE STATEMENTS FROM EV YOUNG WOMEN... meet your mentor. Connect with women in politics. Make YOUR Experience happen. MORE ABOUT EV'S YOUTH MENTORSHIP PROGRAM EV IN ACTION Programs, projects, events and member activities November 28, 2009 Equal Voice AGM November 24, 2009 Let's Give Her a Hand!: Equal Voice celebrates and supports women running for office MORE EV ACTIVITIES AND ACTION IN THE NEWS A digest of news, articles, research & reports about women in politics Nov 23, 2009 Coyle: Honouring Sheila James, and the meaning of life Nov 19, 2009 Life in the Regan household not all politics MORE NEWS ABOUT WOMEN IN POLITICS Prev Next Pause Play PLATINUM SPONSOR MAJOR FUNDING BY BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR More fundamental facts Get involved... in equal voice chapters from coast to coast. Find one near you, or perhaps even start a chapter. Getting to the Gate Online Campaign School: For women of all ages, back- grounds & walks of life interest- ed in running for public office. MORE INFO ON GETTING ELECTED TO PUBLIC OFFICE EV Newsletter Sign up to receive our newsletter Your Name Your Email View archived newsletters Join & Act JOIN - Help create a healthier, more democratic political system in Canada ACT - Participate in an EV Regional or Youth Chapter. Find the chapter nearest you. Donate Equal Voice Needs Your Voice & Your Donation! Click here to make a difference >> Website by: PRIVACY POLICY)
(20091218,www.liberal.ca,http://www.liberal.ca/share_e.aspx?link=http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi,HTTP/1.1 200 OK Cache-Control: private Content-Type: text/html; charset=utf-8 Server: Microsoft-IIS/7.0 X-Powered-By: UrlRewriter.NET 2.0.0 X-AspNet-Version: 2.0.50727 MicrosoftOfficeWebServer: 5.0_Pub X-Powered-By: ASP.NET Date: Fri, 18 Dec 2009 23:17:12 GMT Connection: close Content-Length: 23674  	Liberal Party of Canada » Share with your friends  Liberal Party of Canada Home Your Riding Search Contact Français Michael Ignatieff Introduction Biography Speeches Publications Photos Videos Team Members of Parliament Opposition Critics Senators In Your Community Party Central History Board of directors Election Readiness Commissions En Famille Liberalist Documents In your province/territory Contact Alberta British Columbia Manitoba New Brunswick Newfoundland and Labrador Nova Scotia Ontario Prince Edward Isand Quebec Saskatchewan Contact us Newsroom Blog Media Releases Official Graphics Media Contact Information RSS Newsfeeds Liberal TV Get Involved Join the Party Volunteer Get updates by email Upcoming Events Spread the Word Connect on social media Liberals Abroad Contribute Contribute Get Involved Join the Party | Volunteer | Get Updated | Upcoming Events | Spread the Word | Connect on social media | Liberals Abroad | Contribute Share with your friends You can use this page to tell your friends about the video you just watched and send them a link. Use our address book tool to select your contacts directly from you address book or type in email addresses below, one per line. Invalid addresses will be ignored. We won’t save your information.   Your name:     Your email: Personalize your message: Dear friend, I just finished watching a video on the Liberal Party of Canada’s web site and thought you might like to take a look. You can check out the video by clicking on this link: http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi  Get Involved Connect Facebook | Youtube | Flickr | Twitter | Liblogs Join the Liberal Party Have your say on the future of the party Contribute Give what you can and keep the momentum going Volunteer Sign up and help prepare for the next elections Upcoming Events Find out about upcoming events in your area What's New? Ignatieff hopeful of climate change progress, despite Harper snub in Copenhagen Liberals demand Conservatives stop blocking Afghan Special Committee All partners must join together to save Yarmouth ferry Conservatives squander Canada’s nuclear leadership Liberals demand explanation from Conservatives for Afghan committee no-show Liberals push pension reforms as Flaherty meets with provinces Just the facts: Stephen Harper is a climate change denier Featured Contact Your Riding Search RSS Français © Authorized by the Federal Liberal Agency of Canada, registered agent for the Liberal Party of Canada. M_Ignatieff: The bottom line is Liberals want Canada to commit to verifiable, binding, absolute reductions in line with other developed countries. #cop15 Follow Michael on Twitter >)

Language

Script

import io.archivesunleashed._
import io.archivesunleashed.matchbox._

RecordLoader.loadArchives("/home/nruest/Projects/au/aut-resources/Sample-Data/*.gz", sc)
  .keepValidPages()
  .keepLanguages(Set("en"))
  .map(r => (r.getCrawlDate, r.getDomain, r.getUrl, RemoveHTML(r.getContentString)))
  .saveAsTextFile("/tmp/plain-text-en/")

sys.exit

Results

$ ls /tmp/plain-text-en 
part-00000  part-00001  _SUCCESS

$ head -n 2 /tmp/plain-text-en/part-00000
(20091218,www.equalvoice.ca,http://www.equalvoice.ca/images/images/french/js/images/sponsors/enbridge.jpg,HTTP/1.1 200 OK Connection: close Date: Fri, 18 Dec 2009 23:17:29 GMT Server: Microsoft-IIS/6.0 PICS-Label: (PICS-1.0 "http://www.rsac.org/ratingsv01.html" l by "info@xynapse.ca" on "2004.10.12T16:51-0400" exp "2010.10.12T12:00-0400" r (v 0 s 0 n 0 l 0)) X-Powered-By: ASP.NET Content-Language: en-CA Content-Type: text/html; charset=UTF-8 Equal Voice  Equal Voice HOME | ENGLISH | FRENCH RSS SUBSCRIBE About Us Mission Advisory Board Board of Directors & Staff Programs and Events EV Programs EV Events EV Speaks Out Research and Reports Reports Fundamental Facts Women in Canadian Politics Chapters NCR Ottawa British Columbia Alberta North Alberta South Youth Founders (Toronto) Toronto Municipal Nova Scotia Newfoundland & Labrador Join and Act Become an EV member Take Action Media News Releases In The News Video Photo Gallery Presentations Sponsors Donate Contact Us Contact Us Job Postings EV SPEAKS OUT Latest statements from Equal Voice Nov 19, 2009 Sheila James Fund to be established at Equal Voice Nov 11, 2009 Bulletin d’AVE - automne 2009 MORE STATEMENTS FROM EV YOUNG WOMEN... meet your mentor. Connect with women in politics. Make YOUR Experience happen. MORE ABOUT EV'S YOUTH MENTORSHIP PROGRAM EV IN ACTION Programs, projects, events and member activities November 28, 2009 Equal Voice AGM November 24, 2009 Let's Give Her a Hand!: Equal Voice celebrates and supports women running for office MORE EV ACTIVITIES AND ACTION IN THE NEWS A digest of news, articles, research & reports about women in politics Nov 23, 2009 Coyle: Honouring Sheila James, and the meaning of life Nov 19, 2009 Life in the Regan household not all politics MORE NEWS ABOUT WOMEN IN POLITICS Prev Next Pause Play PLATINUM SPONSOR MAJOR FUNDING BY BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR More fundamental facts Get involved... in equal voice chapters from coast to coast. Find one near you, or perhaps even start a chapter. Getting to the Gate Online Campaign School: For women of all ages, back- grounds & walks of life interest- ed in running for public office. MORE INFO ON GETTING ELECTED TO PUBLIC OFFICE EV Newsletter Sign up to receive our newsletter Your Name Your Email View archived newsletters Join & Act JOIN - Help create a healthier, more democratic political system in Canada ACT - Participate in an EV Regional or Youth Chapter. Find the chapter nearest you. Donate Equal Voice Needs Your Voice & Your Donation! Click here to make a difference >> Website by: PRIVACY POLICY)
(20091218,www.liberal.ca,http://www.liberal.ca/share_e.aspx?link=http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi,HTTP/1.1 200 OK Cache-Control: private Content-Type: text/html; charset=utf-8 Server: Microsoft-IIS/7.0 X-Powered-By: UrlRewriter.NET 2.0.0 X-AspNet-Version: 2.0.50727 MicrosoftOfficeWebServer: 5.0_Pub X-Powered-By: ASP.NET Date: Fri, 18 Dec 2009 23:17:12 GMT Connection: close Content-Length: 23674  	Liberal Party of Canada » Share with your friends  Liberal Party of Canada Home Your Riding Search Contact Français Michael Ignatieff Introduction Biography Speeches Publications Photos Videos Team Members of Parliament Opposition Critics Senators In Your Community Party Central History Board of directors Election Readiness Commissions En Famille Liberalist Documents In your province/territory Contact Alberta British Columbia Manitoba New Brunswick Newfoundland and Labrador Nova Scotia Ontario Prince Edward Isand Quebec Saskatchewan Contact us Newsroom Blog Media Releases Official Graphics Media Contact Information RSS Newsfeeds Liberal TV Get Involved Join the Party Volunteer Get updates by email Upcoming Events Spread the Word Connect on social media Liberals Abroad Contribute Contribute Get Involved Join the Party | Volunteer | Get Updated | Upcoming Events | Spread the Word | Connect on social media | Liberals Abroad | Contribute Share with your friends You can use this page to tell your friends about the video you just watched and send them a link. Use our address book tool to select your contacts directly from you address book or type in email addresses below, one per line. Invalid addresses will be ignored. We won’t save your information.   Your name:     Your email: Personalize your message: Dear friend, I just finished watching a video on the Liberal Party of Canada’s web site and thought you might like to take a look. You can check out the video by clicking on this link: http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi  Get Involved Connect Facebook | Youtube | Flickr | Twitter | Liblogs Join the Liberal Party Have your say on the future of the party Contribute Give what you can and keep the momentum going Volunteer Sign up and help prepare for the next elections Upcoming Events Find out about upcoming events in your area What's New? Ignatieff hopeful of climate change progress, despite Harper snub in Copenhagen Liberals demand Conservatives stop blocking Afghan Special Committee All partners must join together to save Yarmouth ferry Conservatives squander Canada’s nuclear leadership Liberals demand explanation from Conservatives for Afghan committee no-show Liberals push pension reforms as Flaherty meets with provinces Just the facts: Stephen Harper is a climate change denier Featured Contact Your Riding Search RSS Français © Authorized by the Federal Liberal Agency of Canada, registered agent for the Liberal Party of Canada. M_Ignatieff: The bottom line is Liberals want Canada to commit to verifiable, binding, absolute reductions in line with other developed countries. #cop15 Follow Michael on Twitter >)

Anything else to test? Should I through RemoveHttpHeader into the scripts since we see the headers a fair bit in our results?

ianmilligan1 · Aug 1, 2019

I'm going to test with language and full text extraction. That the big stuff?

Yep!

Should I through RemoveHttpHeader into the scripts since we see the headers a fair bit in our results?

Good idea. Behaviour should now always be consistent between ARCs + WARCs.

I'll open up a documentation ticket.

ruebot · Aug 1, 2019

Plain text

Script

import io.archivesunleashed._
import io.archivesunleashed.matchbox._

RecordLoader.loadArchives("/home/nruest/Projects/au/aut-resources/Sample-Data/*.gz", sc)
  .keepValidPages()
  .map(r => (r.getCrawlDate, r.getDomain, r.getUrl, RemoveHTML(RemoveHttpHeader(r.getContentString))))
  .saveAsTextFile("/tmp/plain-text-remove-http/")

sys.exit

Results

$ ls /tmp/plain-text-remove-http    
part-00000  part-00001  _SUCCESS

$ head -n 2 /tmp/plain-text/part-00000
(20091218,www.equalvoice.ca,http://www.equalvoice.ca/images/images/french/js/images/sponsors/enbridge.jpg,Equal Voice  Equal Voice HOME | ENGLISH | FRENCH RSS SUBSCRIBE About Us Mission Advisory Board Board of Directors & Staff Programs and Events EV Programs EV Events EV Speaks Out Research and Reports Reports Fundamental Facts Women in Canadian Politics Chapters NCR Ottawa British Columbia Alberta North Alberta South Youth Founders (Toronto) Toronto Municipal Nova Scotia Newfoundland & Labrador Join and Act Become an EV member Take Action Media News Releases In The News Video Photo Gallery Presentations Sponsors Donate Contact Us Contact Us Job Postings EV SPEAKS OUT Latest statements from Equal Voice Nov 19, 2009 Sheila James Fund to be established at Equal Voice Nov 11, 2009 Bulletin d’AVE - automne 2009 MORE STATEMENTS FROM EV YOUNG WOMEN... meet your mentor. Connect with women in politics. Make YOUR Experience happen. MORE ABOUT EV'S YOUTH MENTORSHIP PROGRAM EV IN ACTION Programs, projects, events and member activities November 28, 2009 Equal Voice AGM November 24, 2009 Let's Give Her a Hand!: Equal Voice celebrates and supports women running for office MORE EV ACTIVITIES AND ACTION IN THE NEWS A digest of news, articles, research & reports about women in politics Nov 23, 2009 Coyle: Honouring Sheila James, and the meaning of life Nov 19, 2009 Life in the Regan household not all politics MORE NEWS ABOUT WOMEN IN POLITICS Prev Next Pause Play PLATINUM SPONSOR MAJOR FUNDING BY BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR More fundamental facts Get involved... in equal voice chapters from coast to coast. Find one near you, or perhaps even start a chapter. Getting to the Gate Online Campaign School: For women of all ages, back- grounds & walks of life interest- ed in running for public office. MORE INFO ON GETTING ELECTED TO PUBLIC OFFICE EV Newsletter Sign up to receive our newsletter Your Name Your Email View archived newsletters Join & Act JOIN - Help create a healthier, more democratic political system in Canada ACT - Participate in an EV Regional or Youth Chapter. Find the chapter nearest you. Donate Equal Voice Needs Your Voice & Your Donation! Click here to make a difference >> Website by: PRIVACY POLICY)
(20091218,www.liberal.ca,http://www.liberal.ca/share_e.aspx?link=http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi,Liberal Party of Canada » Share with your friends  Liberal Party of Canada Home Your Riding Search Contact Français Michael Ignatieff Introduction Biography Speeches Publications Photos Videos Team Members of Parliament Opposition Critics Senators In Your Community Party Central History Board of directors Election Readiness Commissions En Famille Liberalist Documents In your province/territory Contact Alberta British Columbia Manitoba New Brunswick Newfoundland and Labrador Nova Scotia Ontario Prince Edward Isand Quebec Saskatchewan Contact us Newsroom Blog Media Releases Official Graphics Media Contact Information RSS Newsfeeds Liberal TV Get Involved Join the Party Volunteer Get updates by email Upcoming Events Spread the Word Connect on social media Liberals Abroad Contribute Contribute Get Involved Join the Party | Volunteer | Get Updated | Upcoming Events | Spread the Word | Connect on social media | Liberals Abroad | Contribute Share with your friends You can use this page to tell your friends about the video you just watched and send them a link. Use our address book tool to select your contacts directly from you address book or type in email addresses below, one per line. Invalid addresses will be ignored. We won’t save your information.   Your name:     Your email: Personalize your message: Dear friend, I just finished watching a video on the Liberal Party of Canada’s web site and thought you might like to take a look. You can check out the video by clicking on this link: http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi  Get Involved Connect Facebook | Youtube | Flickr | Twitter | Liblogs Join the Liberal Party Have your say on the future of the party Contribute Give what you can and keep the momentum going Volunteer Sign up and help prepare for the next elections Upcoming Events Find out about upcoming events in your area What's New? Ignatieff hopeful of climate change progress, despite Harper snub in Copenhagen Liberals demand Conservatives stop blocking Afghan Special Committee All partners must join together to save Yarmouth ferry Conservatives squander Canada’s nuclear leadership Liberals demand explanation from Conservatives for Afghan committee no-show Liberals push pension reforms as Flaherty meets with provinces Just the facts: Stephen Harper is a climate change denier Featured Contact Your Riding Search RSS Français © Authorized by the Federal Liberal Agency of Canada, registered agent for the Liberal Party of Canada. M_Ignatieff: The bottom line is Liberals want Canada to commit to verifiable, binding, absolute reductions in line with other developed countries. #cop15 Follow Michael on Twitter >)

Language

Script

import io.archivesunleashed._
import io.archivesunleashed.matchbox._

RecordLoader.loadArchives("/home/nruest/Projects/au/aut-resources/Sample-Data/*.gz", sc)
  .keepValidPages()
  .keepLanguages(Set("en"))
  .map(r => (r.getCrawlDate, r.getDomain, r.getUrl, RemoveHTML(RemoveHttpHeader(r.getContentString))))
  .saveAsTextFile("/tmp/plain-text-en-remove-http/")

sys.exit

Results

$ ls /tmp/plain-text-en-remove-http 
part-00000  part-00001  _SUCCESS

$ head -n 2 /tmp/plain-text-en/part-00000
(20091218,www.equalvoice.ca,http://www.equalvoice.ca/images/images/french/js/images/sponsors/enbridge.jpg,Equal Voice  Equal Voice HOME | ENGLISH | FRENCH RSS SUBSCRIBE About Us Mission Advisory Board Board of Directors & Staff Programs and Events EV Programs EV Events EV Speaks Out Research and Reports Reports Fundamental Facts Women in Canadian Politics Chapters NCR Ottawa British Columbia Alberta North Alberta South Youth Founders (Toronto) Toronto Municipal Nova Scotia Newfoundland & Labrador Join and Act Become an EV member Take Action Media News Releases In The News Video Photo Gallery Presentations Sponsors Donate Contact Us Contact Us Job Postings EV SPEAKS OUT Latest statements from Equal Voice Nov 19, 2009 Sheila James Fund to be established at Equal Voice Nov 11, 2009 Bulletin d’AVE - automne 2009 MORE STATEMENTS FROM EV YOUNG WOMEN... meet your mentor. Connect with women in politics. Make YOUR Experience happen. MORE ABOUT EV'S YOUTH MENTORSHIP PROGRAM EV IN ACTION Programs, projects, events and member activities November 28, 2009 Equal Voice AGM November 24, 2009 Let's Give Her a Hand!: Equal Voice celebrates and supports women running for office MORE EV ACTIVITIES AND ACTION IN THE NEWS A digest of news, articles, research & reports about women in politics Nov 23, 2009 Coyle: Honouring Sheila James, and the meaning of life Nov 19, 2009 Life in the Regan household not all politics MORE NEWS ABOUT WOMEN IN POLITICS Prev Next Pause Play PLATINUM SPONSOR MAJOR FUNDING BY BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR BRONZE SPONSOR More fundamental facts Get involved... in equal voice chapters from coast to coast. Find one near you, or perhaps even start a chapter. Getting to the Gate Online Campaign School: For women of all ages, back- grounds & walks of life interest- ed in running for public office. MORE INFO ON GETTING ELECTED TO PUBLIC OFFICE EV Newsletter Sign up to receive our newsletter Your Name Your Email View archived newsletters Join & Act JOIN - Help create a healthier, more democratic political system in Canada ACT - Participate in an EV Regional or Youth Chapter. Find the chapter nearest you. Donate Equal Voice Needs Your Voice & Your Donation! Click here to make a difference >> Website by: PRIVACY POLICY)
(20091218,www.liberal.ca,http://www.liberal.ca/share_e.aspx?link=http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi,Liberal Party of Canada » Share with your friends  Liberal Party of Canada Home Your Riding Search Contact Français Michael Ignatieff Introduction Biography Speeches Publications Photos Videos Team Members of Parliament Opposition Critics Senators In Your Community Party Central History Board of directors Election Readiness Commissions En Famille Liberalist Documents In your province/territory Contact Alberta British Columbia Manitoba New Brunswick Newfoundland and Labrador Nova Scotia Ontario Prince Edward Isand Quebec Saskatchewan Contact us Newsroom Blog Media Releases Official Graphics Media Contact Information RSS Newsfeeds Liberal TV Get Involved Join the Party Volunteer Get updates by email Upcoming Events Spread the Word Connect on social media Liberals Abroad Contribute Contribute Get Involved Join the Party | Volunteer | Get Updated | Upcoming Events | Spread the Word | Connect on social media | Liberals Abroad | Contribute Share with your friends You can use this page to tell your friends about the video you just watched and send them a link. Use our address book tool to select your contacts directly from you address book or type in email addresses below, one per line. Invalid addresses will be ignored. We won’t save your information.   Your name:     Your email: Personalize your message: Dear friend, I just finished watching a video on the Liberal Party of Canada’s web site and thought you might like to take a look. You can check out the video by clicking on this link: http://www.liberal.ca/en/newsroom/liberal-tv/category/56E6B9156BA42F5F_events/4.36363636364/ZSj39F5L1rM~hommage-a-ceux-qui-ont-servi  Get Involved Connect Facebook | Youtube | Flickr | Twitter | Liblogs Join the Liberal Party Have your say on the future of the party Contribute Give what you can and keep the momentum going Volunteer Sign up and help prepare for the next elections Upcoming Events Find out about upcoming events in your area What's New? Ignatieff hopeful of climate change progress, despite Harper snub in Copenhagen Liberals demand Conservatives stop blocking Afghan Special Committee All partners must join together to save Yarmouth ferry Conservatives squander Canada’s nuclear leadership Liberals demand explanation from Conservatives for Afghan committee no-show Liberals push pension reforms as Flaherty meets with provinces Just the facts: Stephen Harper is a climate change denier Featured Contact Your Riding Search RSS Français © Authorized by the Federal Liberal Agency of Canada, registered agent for the Liberal Party of Canada. M_Ignatieff: The bottom line is Liberals want Canada to commit to verifiable, binding, absolute reductions in line with other developed countries. #cop15 Follow Michael on Twitter >)

Looks good to me. I'll wait for a thumbs up from @jrwiebe before I merge.

ruebot · Aug 1, 2019

ruebot approved these changes Aug 1, 2019

View changes

ruebot · Aug 6, 2019

@jrwiebe you good on this one?

jrwiebe · Aug 6, 2019

A-OK

…

On Tue, Aug 6, 2019, 12:09 PM Nick Ruest, ***@***.***> wrote: @jrwiebe <https://github.com/jrwiebe> you good on this one? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#335?email_source=notifications&email_token=ABDJJDOD526E3WZQIQMGCV3QDGV2XA5CNFSM4IIAIVMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3V2K5Y#issuecomment-518759799>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDJJDOAZS5B3IFNCFR4QBLQDGV2XANCNFSM4IIAIVMA> .

Make ArchiveRecord.getContentBytes consistent,#334

Loading status checks…

f2bfa3b

ianmilligan1 requested a review from ruebot Jul 30, 2019

Merge branch 'master' into issue-334

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

40bc126

ianmilligan1 referenced this pull request Aug 1, 2019
Open
Document RemoveHttpHeader and build into some of the documentation #130

Merge branch 'master' into issue-334

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

5c1637b

ruebot deleted the issue-334 branch Aug 6, 2019

archivesunleashed/aut

Make ArchiveRecord.getContentBytes consistent, Resolve #334 #335

Make ArchiveRecord.getContentBytes consistent, Resolve #334 #335

ianmilligan1 commented Jul 30, 2019

ianmilligan1 requested a review from ruebot Jul 30, 2019

This comment has been minimized.

codecov-io commented Jul 30, 2019 •

edited

This comment has been minimized.

ruebot commented Aug 1, 2019

This comment has been minimized.

ruebot commented Aug 1, 2019

This comment has been minimized.

ianmilligan1 commented Aug 1, 2019

ianmilligan1 referenced this pull request Aug 1, 2019

Document RemoveHttpHeader and build into some of the documentation #130

This comment has been minimized.

ruebot commented Aug 1, 2019

ruebot approved these changes Aug 1, 2019

View changes

This comment has been minimized.

ruebot commented Aug 6, 2019

This comment has been minimized.

jrwiebe commented Aug 6, 2019

ruebot merged commit `1818596` into master Aug 6, 2019
3 checks passed

3 checks passed

ruebot deleted the issue-334 branch Aug 6, 2019

archivesunleashed/aut

Join GitHub today

Make ArchiveRecord.getContentBytes consistent, Resolve #334 #335

Conversation

ianmilligan1 commented Jul 30, 2019

What does this Pull Request do?

How should this be tested?

Additional Notes:

Interested parties

ianmilligan1 requested a review from ruebot Jul 30, 2019

This comment has been minimized.

codecov-io commented Jul 30, 2019 • edited

Codecov Report

This comment has been minimized.

ruebot commented Aug 1, 2019

This comment has been minimized.

ruebot commented Aug 1, 2019

Plain text

Language

This comment has been minimized.

ianmilligan1 commented Aug 1, 2019

ianmilligan1 referenced this pull request Aug 1, 2019

Document RemoveHttpHeader and build into some of the documentation #130

This comment has been minimized.

ruebot commented Aug 1, 2019

Plain text

Language

ruebot approved these changes Aug 1, 2019 View changes

This comment has been minimized.

ruebot commented Aug 6, 2019

This comment has been minimized.

jrwiebe commented Aug 6, 2019

Hide details View details ruebot merged commit 1818596 into master Aug 6, 2019 3 checks passed

3 checks passed

ruebot deleted the issue-334 branch Aug 6, 2019

codecov-io commented Jul 30, 2019 •

edited

ruebot approved these changes Aug 1, 2019

View changes

ruebot merged commit `1818596` into master Aug 6, 2019
3 checks passed