Logfile with non-UTF-8 character causing error in API stats reporting

Description

When trying to create the November API stats report, I encountered the following error:

<pre>
dpla@api-prod2:/srv/www/api$ cat var/log/production.log.stats-for-november var/log/production.log-2014110[89]* var/log/production.log-201411[123]* | bundle exec bin/get-api-stats.rb http://repo-prod1:5984/dpla_api_auth/_design/api_auth_utils/_view/find_by_owner "2ea7d65b8aec3f7e7704082f1f2de9f4, 00529aa4925f81abc77f752ef050e690, 99b7a3ceccb20241f5d525001df8a7bf, aa22c5ec71f95032dbcba4afc2041deb, 66b24c8ffcca9b7708d33ecddaec1d2d, 4df12e9190d635653a718c95b37f00de, 078f539064feb6684b184ec133ac1c77, 1e29c7dbacf9e5a6a5537174022fc1c1, aa22c5ec71f95032dbcba4afc2041deb" > ~/201411-stats-api-prod2.tab
2014-11-01
2014-11-02
2014-11-03
2014-11-04
2014-11-05
2014-11-06
2014-11-07
bin/get-api-stats.rb:74:in `=~': invalid byte sequence in UTF-8 (ArgumentError)
from bin/get-api-stats.rb:74:in `block in analyze_file'
from bin/get-api-stats.rb:73:in `each_line'
from bin/get-api-stats.rb:73:in `analyze_file'
from bin/get-api-stats.rb:19:in `main'
from bin/get-api-stats.rb:124:in `<main>'
</pre>

Mark B. identified the probably source of the error as a logfile with a non-UTF-8 character. <pre>get-api-stats.rb</pre> should fixed so that it can deal with non-UTF-8 characters.

Kenny and Dan need the reports by Friday, so this is high priority.

Activity

Show:
Mark Matienzo
December 6, 2014, 7:46 AM

Audrey wrote on "dpla/platform#33":https://github.com/dpla/platform/pull/33#issuecomment-65846813 :

> After further inspection, I realize that the only logfile that required this fix was actually not a logfile at all. Rather, it was a special record that markbreedlove made to account for the rs to aws changeover. Since we are not likely to ever again encounter a file such as this, and by extension an error such as this, I wonder if it is worth changing the codebase at all.

Done

Assignee

Audrey Altman

Reporter

Audrey Altman

Labels

None

Priority

High