Using sed to modify the date/time in a gpx file

boughtonp · 06-11-2021, 06:32 AM

Quote:

Originally Posted by Johng

I have some gpx files that I want to modify the date/time.

Why?

What is the problem you're trying to solve by adding/subtracting twelve hours?

The software world is infested with bugs created by people who have tried to ignore the nature of what they're dealing with by using brittle hacks, instead of simply solving the true issue.

Johng · 06-11-2021, 07:23 PM

To shruggy
Yes I tried your script in #4. The output printed in the terminal twice with no line breaks. The date and times were not changed. Between versions there was an error message:

Quote:

== Test.gpx ==
./shruggy: line 17: dateutils.dadd: command not found
Test.gpx:2.11: xmlns: URI GPX/1/1 is not absolute
<gpx xmlns="GPX/1/1" version="1.1" creator="JGxxxxx">
^
Test.gpx:2.11: xmlns: URI GPX/1/1 is not absolute
<gpx xmlns="GPX/1/1" version="1.1" creator="JGxxxxx">

To MadeInGermany
Your perl command in post 15 did what I requested in my first post, thank you.

And thank you all who offered advice

ondoho · 06-12-2021, 03:40 AM

Quote:

Originally Posted by boughtonp

Why?

Sounds like you won't be getting an answer:

Quote:

Originally Posted by Johng

And thank you all who offered advice

shruggy · 06-12-2021, 03:45 AM

Quote:

./shruggy: line 17: dateutils.dadd: command not found

Code:

sudo apt install dateutils

Quote:

Test.gpx:2.11: xmlns: URI GPX/1/1 is not absolute
<gpx xmlns="GPX/1/1" version="1.1" creator="JGxxxxx">

Code:

sed -i '/xmlns=/s|GPX/1/1|http://www.topografix.com/&|' Test.gpx

Johng · 06-12-2021, 06:56 PM

To ondoho:
Appologies for not answering Why?

One of my GPS devices suffered the GPS Week Number Rollover (WNRO) effect. For affected GPS devices, after the rollover, an incorrect date and time is displayed. This incorrect time is also be used to timestamp track logs, compute sunrise and sunset, and other functions that rely upon the correct date and time. However, the positioning accuracy is not be affected. The device continues to deliver the same positioning performance as before the rollover.

So I decided to write a script that would add 7168 days to the date when downloading the Garmin data. This has worked well. However, UTC is not as convenient as local Standard Time, and this is why I sought a simple way to edit the downloaded time in the one operation

To shruggy:.
I installed 'dateutils' and tried the script again. The script changes the date but not the time; the output is a single line. Adding http://www.topografix.com/ to the header removes the error message; this addition has not been a requirement of any other software.

Johng · 06-12-2021, 07:37 PM

To MadeInGermany

Your perl command that modified the time was very concise, would you have a perl command adding 7168 days to the date as well:

Quote:

becoming:

Quote:

boughtonp · 06-13-2021, 08:38 AM

Quote:

Originally Posted by Johng

However, UTC is not as convenient as local Standard Time, and this is why I sought a simple way to edit the downloaded time in the one operation

Thought so. The Z at the end of ISO 8601 formatted timestamp indicates UTC (zero offset) - if you're modifying the time, you probably want to either remove that Z to make it a non-specified local timestamp, or switch it to a +12 (to indicate the time is twelve hours ahead of UTC i.e. New Zealand Standard Time).

Alternatively, just specify the output timezone and any ISO 8601 compliant tool will give you the correct NZ time:

Code:

$ TZ=NZ date --date="2001-10-24T03:03:44.000Z + 7168 days"
Wed  9 Jun 15:03:44 NZST 2021
$ TZ=NZ date --date="2001-10-24T03:03:44.000Z + 7168 days" -Is
2021-06-09T15:03:44+12:00

There should be similar timezone-related options with Perl/etc.

(I'm still not clear on why/when the subtraction of 12 hours should happen.)

Johng · 06-13-2021, 08:46 PM

Thank you boughtonp

Code:

$ TZ=NZ date --date="2001-10-24T03:03:44.000Z + 7168 days" -Is
2021-06-09T15:03:44+12:00

is great, modifying the Date and Time. I have not yet worked out a simple way to apply it to a line like:

Quote:

Quote:

(I'm still not clear on why/when the subtraction of 12 hours should happen.)

After correcting the date, I was treating the hour value as a number, not as time.

shruggy · 02-14-2022, 02:19 PM

Revisiting this old thread to show the Python solution using gpxpy.

Code:

#!/usr/bin/python3
import argparse
import gpxpy
from datetime import timedelta, timezone

prog = 'gpxpy'
description = 'Adjust timestamps'
parser = argparse.ArgumentParser(prog=prog, description=description)
parser.add_argument('infile', type=str, nargs='+',
                    help='input GPX file')
args = parser.parse_args()
gpx_files = args.infile

tz = timezone(timedelta(hours=12))

for gpx_file in gpx_files:
    with open(gpx_file) as f:
        gpx = gpxpy.parse(f)
        for track in gpx.tracks:
            for segment in track.segments:
                for point in segment.points:
                    point.time = point.time + timedelta(days=7168)
                    point.time = point.time.astimezone(tz=tz)
                    # This would do it for the system local timezone
                    #point.time = point.time.astimezone()
        print(gpx.to_xml())

The Bash script from #4, now with timezone as well.

Code:

#!/bin/bash
# dateutils have non-standard names on Debian-based systems
. /etc/os-release
[[ $ID_LIKE =~ debian ]] &&
  dateadd=dateutils.dadd ||
  dateadd=dateadd

# xmlstarlet is named xml in FreeBSD, PCLinuxOS, Void.
# (it's both xmlstarlet and xml in Alpine, Arch, openSUSE)
if [[ $(uname) =~ BSD ]] || [[ -f /etc/pclinuxos-release ]] || [[ $ID == void ]]
then xmlstarlet=xml
else xmlstarlet=xmlstarlet
fi

for f in *.gpx
do
  echo "== $f =="
  # xmlstarlet is picky about namespaces
  grep -q 'xmlns=' "$f" && ns=_: || ns=

  timestamps=(
    $($xmlstarlet sel -t -v //${ns}trkpt/${ns}time -n "$f" | $dateadd -zNZ 7168d)
  )
  args=
  for k in "${!timestamps[@]}"
  do
    args+=" -u //${ns}trkpt[$((k+1))]/${ns}time"
    args+=" -v ${timestamps[$k]}+12:00"
  done
  $xmlstarlet ed $args "$f"
  echo =============
done