Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filter/keep by http status to RecordLoader class #315

Closed
greebie opened this issue Apr 5, 2019 · 0 comments

Comments

@greebie
Copy link
Contributor

commented Apr 5, 2019

Is your feature request related to a problem? Please describe.

Now that we have http status as part of the ArchiveRecord class, it may make sense to include methods to filter and or keep record with particular status codes.

Describe the solution you'd like
Something like:

RecordLoader.loadArchives(path, sc)
  .keepValidPages()
  .filterHttpStatus(Set('404', '410'))

and same for .keepHttpStatus

Describe alternatives you've considered
@edsu also suggested that users may already expect certain Http Statuses (e.g. 410) to be removed from .keepValidPages() as well.

Additional context

See discussion in ArchivesUnleashed Slack Aut Channel.

@ruebot ruebot self-assigned this Aug 17, 2019

ruebot added a commit that referenced this issue Aug 17, 2019

Add keep and discard by http status.
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315

ruebot added a commit that referenced this issue Aug 17, 2019

Add keep and discard by http status.
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315

ruebot added a commit that referenced this issue Aug 17, 2019

Add keep and discard by http status.
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315

ianmilligan1 added a commit that referenced this issue Aug 17, 2019

Add keep and discard by http status. (#347)
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.