• 1 Post
  • 35 Comments
Joined 2 年前
cake
Cake day: 2023年7月17日

help-circle
  • This process is not triggered by any external events.

    Every ten minutes, an internal background job activates. Its function is to scan the database for any RawLocationPoints that haven’t been processed yet. These unprocessed points are then batched into groups of 100, and each batch is sent as a message to be consumed by the stay-detection-queue. This process naturally adds to the workload of that queue.

    However, if no new location data is being ingested, once all RawLocationPoints have been processed and their respective flags set, the stay-detection-queue should eventually clear, and the system should return to a idle state. I’m still puzzled as to why this initial queue (stay-detection-queue) is exhibiting such slow performance for you, as it’s typically one of the faster steps.


  • Thank you for testing Reitti. 🙏

    It depends on two key requirements for Reitti:

    1. First, it finds all photos from Immich taken on the day you selected.
    2. Then, it filters these photos based on the selected map bounds, using the embedded EXIF geolocation data (where the photo was shot).

    If the EXIF data does not contain geolocation information, we currently cannot display those photos because their placement on the map cannot be determined.

    Could you please verify in Immich if the expected photo has its location in the metadata? If it is available there, then the issue might lie in how Reitti is parsing that specific data.


  • That’s good, but I still question why it is so slow. If you receive these timeout exceptions more often, at some point the data will cease to be analyzed.

    I just re-tested it with multiple concurrent imports into a clean DB, and the stay-detection-queue completed in 10 minutes. It’s not normal for it to take that long for you. The component that should take the most time is actually the merge-visit-queue because this creates a lot of stress for the DB. This test was conducted on my laptop, equipped with an AMD Ryzen™ 7 PRO 8840U and 32GB of RAM.




  • Hmm, I had hoped you say something like a Raspberry PI :D

    But this should be enough to have it processed in a reasonable time. What I do not understand in the moment is, that the filesize should not affect it in any way. When importing it 100 Geopoints are bundled, send to RabbitMQ. From there we retrieve them, do some filtering and save them in the database. Then actually nothing happens anymore until the next processing run is triggered.

    But this than works with the PostGis DB and not with the file anymore. So the culprit should be there somewhere. I will try to insert some fake data into mine and see how long it takes if i double my location points.


  • Thanks for the information. I will try to recreate it locally. In my testing I used a 600MB file and this took maybe 2 hours to process on my server. It is one of these ryzen 7 5825U. Since Reitti tries to do these analysis on multiple cores we start it with 4 to 16 Threads when processing. But the stay detection breaks when doing it that way, so it is locking per user to handle that. If now one of them takes a long time the others will break eventually. They will get resheduled 3 times until rabbitmq gives up.

    On what type of system do you run it?

    I will add some switches so it is configurable how many threads are opened and add some log statements to print out the duration it took for a single step.












  • I was thinking about that, but the main problem is that we do not store all the data which comes in.

    If we ingest data from an app, I am pretty sure that the quality of the data is actually usable. But for example if we import an Records.json from Google Takeout. The quality of the earlier years is somewhat sketchy. For this we filter out some points like travelling with over 2000 km/h, sudden direction changes etc and they are lost forever. At least for Reitti they are unknown.

    The feature would need a lot of explanation why the data we export is not the same we import.That is the reason I did not implemented it even if it would come in handy for testing stuff. Handling GPX files is a pita …


  • Maybe the wording is confusing in the Readme. Reitti will try to fetch the data from a configured photon instance first, if this does not return anything and you have Geocoding services configured, it will try them. There is actually no switch for hybrid mode or only local. It depends on what is configured.

    Photon Only: you have only photon configured and under Settings > Gecoding you deleted or disable every available service. Hybrid Mode: Photon is configured and under Settings > Geocoding there are Services available. That es is the one I use. Having Photon with the data for Germany and all the rest is handled by Geoapify.com.

    External Only: You dropped Photon from the docker-compose file and only rely on services under Settings > Geocoding

    If you do not configure anything, then Reitti will skip Geocoding and only display Unknown Place.

    I will update the Readme to make that clear.