I have a directory of .gz files that contain raw boomerang beacon data. I need those files enumerated, extract the single text file from each .gz file. Read each JSON object from that file, could be one or more JSON objects per file and rehydrate the resource timing part of the file to be the uncompressed boomerang resource timing data. I need all of this to happen as new files are written. I suppose if it went to sleep and woke up every min that would be ok but ideally there would be some kind of file watch that would kick things off as files are written out.
We\'ve already done some basic work rehydrating the data using a js library that did the compression, it has a decompression companion script.
Details about the compression: [login to view URL]
Decompression companion script can be found: ([login to view URL])
I\'m happy to provide the existing code if requested. All it does it read a text file, rehydrate the resource timing data and write out to complete version of the file. It doesn\'t enumerate thru the directories nor does it extract the file from the .gz.
I could obviously finish this up myself but I\'m not a developer and have other things that I should be working on. Hoping someone can finish this off for me. Also.... The original source of the data is in an s3 bucket, if the app would pull files directly form there as they are written that would be great as well but not a requirement.
This app will run in a windows environment, so... if you don\'t pull from an S3 bucket you need to read from a windows file system and write the output back to that same windows file system but in a different directory.
Let me know what questions you have.
Hi,
I am experienced nodejs developer, i can extract from .gz from s3 and read json data and write new files. It will be windows compatible
Contact me if you are interested
Hi,
I have expertise in NodeJS, JSON, file manipulations and linux command line.
I can write the exact script you want.
Let's discuss the details, and we can start working on this.
Regards,
Hassan