Update NM scrapers to use new NM legislature website

Description

The New Mexico legislature has a new site at http://www.nmlegis.gov:8080/. As per an email exchange with Seth Morris at the NM legislature site, the NM legislature will switch to its new site on May 1. The old site will no longer be available at that time. Open States probably should replace the scrapers before May 1 and then switch the URL to remove port 8080 on the new scrapers when the new site replaces the old one.

My questions:

(1) When will the old site currently at http://www.nmlegis.gov/ be removed? (2) Will the new site take that address, without the port 8080 that accessing the new site currently requires? (3) If the new site will be available at http://www.nmlegis.gov, do you have a target date for the change and will the new one continue to be available at http://www.nmlegis.gov:8080/ for a time period?

Seth's answers:

1. We will be replacing the old site with the new one on the weekend of May 1st.

2. Yes, the new website will be at www.nmlegis.gov and the old one will no longer be available.

3. We are making the switch over the weekend of May 1st. We will not keep the :8080 address up but the urls will not change except the removal of :8080. For example,

http://www.nmlegis.gov:8080/Calendar/Whats_Happening will become

http://www.nmlegis.gov/Calendar/Whats_Happening

Activity

Show:
Clifton McIntosh
April 12, 2016, 11:58 AM

I verified with Seth Morris that the URLs that openstates uses for document retrieval will remain the same. These include ftp://www.nmlegis.gov/other/ and the subdirectories at http://www.nmlegis.gov/Sessions/. Those are all used in bill scraping.

Other than having port 8080 in the URLs, changes should be in place for when the old site is removed. The following pull requests, all of which have been merged into master, cover the work that was needed:

https://github.com/sunlightlabs/openstates/pull/1084
https://github.com/sunlightlabs/openstates/pull/1081
https://github.com/sunlightlabs/openstates/pull/1074
https://github.com/sunlightlabs/openstates/pull/1068

How does openstates want to handle the port removal on the weekend of May 1? I can make the changes and submit a PR after the site switch happens, or I can write a function that checks the date and either uses the port number (before May 1) or doesn't use the port number (on May 1 and afterwards). That would rely on the NM legislature's switch happening as scheduled, though. That seems a little risky. If you want me to do it that way, please let me know, and I will be happy to do it.

Clifton McIntosh
September 18, 2016, 8:50 PM

This has been completed and should be closed.

Assignee

Unassigned

Reporter

Clifton McIntosh

Priority

Normal
Configure