Delivered-To: igs-dcwg@igscb.jpl.nasa.gov Message-ID: <7CB5F45D9E56D511836F0002A537DD6992C67D@s5-sid-r1.nrcan.gc.ca> From: "Schmidt, Michael" To: 'Carey Noll' , "Angelyn W. Moore" Cc: igs-dcwg@igscb.jpl.nasa.gov Subject: [IGS-DCWG-34] RE: [IGS-DCWG-29] Re: [IGS-DCWG-27] Any DCs accepting .gz RINEX? Date: Thu, 18 Nov 2004 15:01:14 -0800 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: owner-igs-dcwg Precedence: bulk ****************************************************************************** IGS-DCWG Mail 18 Nov 15:11:33 PST 2004 Message Number 34 ****************************************************************************** Author: Michael Schmidt Hi everybody To partially answer Carey's question of .Z vs. .gz vs. .zip files, here = is a brief comparison of compression of a 1 hr, 1Hz file (on Solaris): -rw-r--r-- 1 wcdaop gpsusers 933624 Nov 18 21:49 bcdl320a.04d -rw-r--r-- 1 wcdaop gpsusers 297759 Nov 18 21:49 bcdl320a.04d.gz -rw-r--r-- 1 wcdaop gpsusers 291365 Nov 18 21:48 bcdl320a.04d.Z -rw-r--r-- 1 wcdaop gpsusers 297884 Nov 18 22:11 bcdl320a.04d.zip -rw-r--r-- 1 wcdaop gpsusers 2583764 Nov 17 17:55 bcdl320a.04o -rw-r--r-- 1 wcdaop gpsusers 898794 Nov 17 17:55 bcdl320a.04o.gz -rw-r--r-- 1 wcdaop gpsusers 961869 Nov 17 17:55 bcdl320a.04o.Z -rw-r--r-- 1 wcdaop gpsusers 898919 Nov 18 22:10 bcdl320a.04o.zip Note that the .gz and .zip compressions of the .04o file is slightly = better than the .Z compression but that the opposite is true for the .04d = file. Wrt which one to use, my 2cents(Cdn) worth: a) it would not appear that there is a huge space saving of one over = the other {from this limited, though highly scientific test ... ahem .. :>) = } b) it would be appropriate to examine which format is best for our user base, specifically the none-IGS user base, and that would include the = use of the Hatanaka compression (I can't tell you how many phone calls / = emails I have had on that one ...); for the purpose of this discussion I am = assuming that most core IGS data / analysis centres can handle any of the above, including Hatanaka. However the same cannot necessarily be said for the non-IGS users: - commercial programs can uncompress any of the above (except Hatanaka) = but cannot necesssarily re-compress them in that format (based on examining Winzip's web page). - what format can commercial GPS packages handle? In other words how = can we make it easier for the non-expert to handle IGS data streams and thus encourage / increase use of our products? - is it reasonable to continue with the Hatanaka compression given that there are few commercial applications that can handle it? - I believe = some of the vendor software packages can but is this universal? - should keep in mind that many people do not know how to use DOS = programs (including the latest batch of grad students we've had) so in the = B.Gates world, utilities really have to be Windows based; I believe the gzip utilities are only available for DOS, not Windows (???), similarly = Hatanaka (except as noted above). c) my suggestion would be the following: i) status quo for the time being; ii) do a little bit of research to see what the optimal compression = scheme would be and what tools are available from / supported by the GNSS = vendors and at that point make any adjustments as required. iii) roll out the change(s) once we are sure that these will address = current / future needs. iv) don't bother converting old data files, but leave that up to = individual data centres. I tend to agree with Angie that .Z may become a "legacy" format and = that the gzip format will probably become more widely used. I would be curious = to hear what the common compression formats outside of N. America are, especially if different from the above. end of 2 cents - hope I have not overcharged :>) cheers Mike PS if we had a compression scheme as effective as the proprietaty = Trimble T00 format all of this would be a mute point .... ------------------------------------------------------------------------= ---- Michael Schmidt, P.Eng. MSchmidt@nrcan.gc.ca TEL +1 250-363-6760 FAX +1 250-363-6565 Mobile +1 250-920-6128 Geological Survey of Canada Commission g=E9ologique du Canada Natural Resources Canada Ressources naturelles Canada PO Box 6000 B.P. 6000 Sidney BC Canada V8L 4B2 Sidney (C.B.) Canada V8L 4B2 Street: Rue: 9860 West Saanich Rd 9860 West Saanich Rd North Saanich BC North Saanich (C.B.) Canada V8L 3S1 Canada V8L 3S1 Government of Canada Gouvernement du Canada www.pgc.nrcan.gc.ca ------------------------------------------------------------------------= ---- > -----Original Message----- > From: owner-igs-dcwg@igscb.jpl.nasa.gov=20 > [mailto:owner-igs-dcwg@igscb.jpl.nasa.gov] On Behalf Of Carey Noll > Sent: Thursday, November 18, 2004 1:10 PM > To: Angelyn W. Moore > Cc: igs-dcwg@igscb.jpl.nasa.gov > Subject: [IGS-DCWG-29] Re: [IGS-DCWG-27] Any DCs accepting .gz RINEX? >=20 >=20 > ************************************************************** > **************** > IGS-DCWG Mail 18 Nov 13:15:21 PST 2004 Message Number 29 > ************************************************************** > **************** >=20 > Author: Carey Noll >=20 > Angie, >=20 > CDDIS can certainly do a translation of a file from .gz to=20 > Z as long as we know who is doing it ahead of time so the=20 > scripts can be modified of course. There may also be some=20 > delay in data availability by adding this extra step of=20 > course (minor). >=20 > A large archive like CDDIS does not really want to re-do=20 > the entire archive in another compression scheme! Especially=20 > since we are trying to get the archive up and running on a=20 > new server. I don't see the benefit of doing this, but I am=20 > probably wearing blinders. >=20 > Carey. >=20 > = +=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D= +=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D > +=3D+=3D+=3D+=3D+=3D > ++ > Ms. Carey Noll Manager, Crustal Dynamics Data Information=20 > System (CDDIS) > Secretary, ILRS=20 > Central Bureau > Code 922 E-mail: =20 > Carey.Noll@nasa.gov > NASA GSFC Voice:=20 > (301) 614-6542 > Greenbelt, MD 20771 Fax:=20 > (301) 614-6099 > USA WWW: =20 > http://cddisa.gsfc.nasa.gov/cddis_welcome.html >=20 > = +=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D= +=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D > +=3D+=3D+=3D+=3D+=3D > ++ >=20