-
am0ebe authored
> only full raw data in ecotrack/all/01-raw > all other ecotrack/{I,II,III} only link to this -> should speed up git and reduce total size and redundancy > getDates.sh: add earliest latest date for all units write to file add to dataSync ------------------------------- All the squashed old commits below: ------------------------------- commit baafb4ad (HEAD -> dev, tag: ecotrack-data, tag: ecolux-data, origin/dev, master) Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 18 17:57:32 2021 +0100 recalc *ALL* data + ecotrackimprovement + ... > see below unknown-tags.sh: fix error message. always call both functions and write to files (tags_info and tags_unknowntags) update-gitignore.sh: change name dataSync: calc mean temperature for all 4 sensors filter temp below threshold (cur=12°C) datafilter: fix format bug where zeros where prepended to species col duplicate func make_rel_abs_path (in dataSync and dataFilter) to avoid import error due to circular dependency commit 84cafbb7 Author: am0ebe <am0ebe@gmx.de> Date: Wed Mar 17 18:00:05 2021 +0100 fix config. filter: infer proj from filename. add ectrack data I,II,III. start temp: calc mean for commit 68f4680e Author: am0ebe <am0ebe@gmx.de> Date: Tue Mar 16 15:28:27 2021 +0100 continue conf.json and datasync/filter implementation. filter: derive proj for blox from header commit b419a53b Author: am0ebe <am0ebe@gmx.de> Date: Mon Mar 15 14:48:05 2021 +0100 ignore .csv, add ecotrack data, parse conf.json > add blox and project configuration. commit 99322855 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 18:29:56 2021 +0100 recalc *ALL*. adjust unknowntags script > create tagInfo and unknown-tags automatically from datasync commit a43a6e59 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 16:01:30 2021 +0100 put blox-configuration into file. unknown-tags.sh takes dir as arg and is beong called properly from dataSync commit 75f647a9 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 14:22:05 2021 +0100 add ecotrack data. rename data dirs commit ac955cb8 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 13:35:30 2021 +0100 add ecotrack support | fix time bug > see below fix daylightsavingtimes bug: > set timedelta to hours not days ecotrack support: > 4 sensors per unit > merge on unit and habitat and time! > sensors log time once every 5 minutes. find closes trackdetection > calcHabitat() to determine what habitat depending on x,y and PROJ var > initTempData() get temp data and store in list of list of tuples -> tempData[unit][habitat] returns (time,temp) > mergeTemp
am0ebe authored> only full raw data in ecotrack/all/01-raw > all other ecotrack/{I,II,III} only link to this -> should speed up git and reduce total size and redundancy > getDates.sh: add earliest latest date for all units write to file add to dataSync ------------------------------- All the squashed old commits below: ------------------------------- commit baafb4ad (HEAD -> dev, tag: ecotrack-data, tag: ecolux-data, origin/dev, master) Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 18 17:57:32 2021 +0100 recalc *ALL* data + ecotrackimprovement + ... > see below unknown-tags.sh: fix error message. always call both functions and write to files (tags_info and tags_unknowntags) update-gitignore.sh: change name dataSync: calc mean temperature for all 4 sensors filter temp below threshold (cur=12°C) datafilter: fix format bug where zeros where prepended to species col duplicate func make_rel_abs_path (in dataSync and dataFilter) to avoid import error due to circular dependency commit 84cafbb7 Author: am0ebe <am0ebe@gmx.de> Date: Wed Mar 17 18:00:05 2021 +0100 fix config. filter: infer proj from filename. add ectrack data I,II,III. start temp: calc mean for commit 68f4680e Author: am0ebe <am0ebe@gmx.de> Date: Tue Mar 16 15:28:27 2021 +0100 continue conf.json and datasync/filter implementation. filter: derive proj for blox from header commit b419a53b Author: am0ebe <am0ebe@gmx.de> Date: Mon Mar 15 14:48:05 2021 +0100 ignore .csv, add ecotrack data, parse conf.json > add blox and project configuration. commit 99322855 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 18:29:56 2021 +0100 recalc *ALL*. adjust unknowntags script > create tagInfo and unknown-tags automatically from datasync commit a43a6e59 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 16:01:30 2021 +0100 put blox-configuration into file. unknown-tags.sh takes dir as arg and is beong called properly from dataSync commit 75f647a9 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 14:22:05 2021 +0100 add ecotrack data. rename data dirs commit ac955cb8 Author: am0ebe <am0ebe@gmx.de> Date: Thu Mar 11 13:35:30 2021 +0100 add ecotrack support | fix time bug > see below fix daylightsavingtimes bug: > set timedelta to hours not days ecotrack support: > 4 sensors per unit > merge on unit and habitat and time! > sensors log time once every 5 minutes. find closes trackdetection > calcHabitat() to determine what habitat depending on x,y and PROJ var > initTempData() get temp data and store in list of list of tuples -> tempData[unit][habitat] returns (time,temp) > mergeTemp
Code owners
Assign users and groups as approvers for specific file changes. Learn more.
dataViz.py 1.59 KiB
#!/usr/bin/python3
#
# VIZ (ET/ST)
###################
# sa: ../schrebatrondata/08_alpha_rep1_rep2/02_data_inspect/ana.py
# preq: should work on one huge table with all data!
# > amount PatchCrossingEvents - Time -> activity histogram per unit/rep
# > detection_duration_plot
# > timespan_id_last_seen foreach tag
# > timespan_id_last_seen foreach group (boxplot)
# > Graphs per unit:
# * all
# * species group A ..
# > cluster?
#
# CAMTRACK VIZ:
# > Total distance of movement
# > Minimum and maximum speed
# > If possible: turning angle
#
# Processing
##############
# > parse CSV fields neccessary?
# movement per tagID: #detections
# coolOffPhase = 1s?
# interPatch crossings
# most/least movement
# tag not detected (dead,malfunct,lost)
# ... is there a complete tagList somewhere?
# sensors with most detection
# sensors with no/least detection
# need patchConfig.json ?
#from 3k data scripts:
###############
# > collect all tagIDs in list
# > get species - tagID mapping -> xtract from .ods file
# > sort by tagID
# for element in sorted(csvdata, key=attrgetter('tagID')):
# import json
# json.load("patchconfig.json")
speciesDict = {
"C" : "Carabus nemoralis",
"N" : "Nebria brevicollis",
"PM" : "Pterochstichus melanarius",
"A" : "Abax parallelus",
"P" : "Pyrrhocoris apterus",
"CS" : "Coccinella septempunctata",
"CA" : "Cattalus fuscipes",
"PO" : "Poecilus spec.",
"PR" : "Pseudophonus rufipes",
"HL" : "Harpalus latus",
"H" : "Harpalus spec.",
"PC" : "Pterochtichus cristatus",
"CG" : "Carabus granulatus"
"PQ" : "Pterostichus quadrifoveolatus"
}