Skip to content
Snippets Groups Projects
  • am0ebe's avatar
    4174320b
    !!!REWRITE HISTORY, SAVE SPACE FOR ECOTRACK, SEE FULL COMMIT MSG FOR DETAILS!!! · 4174320b
    am0ebe authored
    > only full raw data in ecotrack/all/01-raw
    > all other ecotrack/{I,II,III} only link to this
    -> should speed up git and reduce total size and redundancy
    
    > getDates.sh:
    	add earliest latest date for all units
    	write to file
    	add to dataSync
    
    -------------------------------
    All the squashed old commits below:
    -------------------------------
    
    commit baafb4ad (HEAD -> dev, tag: ecotrack-data, tag: ecolux-data, origin/dev, master)
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 18 17:57:32 2021 +0100
    
        recalc *ALL* data + ecotrackimprovement + ... > see below
    
        unknown-tags.sh: fix error message. always call both functions and write to files (tags_info and tags_unknowntags)
        update-gitignore.sh: change name
        dataSync:
          calc mean temperature for all 4 sensors
          filter temp below threshold (cur=12°C)
        datafilter:
          fix format bug where zeros where prepended to species col
          duplicate func make_rel_abs_path (in dataSync and dataFilter) to avoid import error due to circular dependency
    
    commit 84cafbb7
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Wed Mar 17 18:00:05 2021 +0100
    
        fix config. filter: infer proj from filename. add ectrack data I,II,III. start temp: calc mean for
    
    commit 68f4680e
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Tue Mar 16 15:28:27 2021 +0100
    
        continue conf.json and datasync/filter implementation. filter: derive proj for blox from header
    
    commit b419a53b
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Mon Mar 15 14:48:05 2021 +0100
    
        ignore .csv, add ecotrack data, parse conf.json > add blox and project configuration.
    
    commit 99322855
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 18:29:56 2021 +0100
    
        recalc *ALL*. adjust unknowntags script > create tagInfo and unknown-tags automatically from datasync
    
    commit a43a6e59
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 16:01:30 2021 +0100
    
        put blox-configuration into file. unknown-tags.sh takes dir as arg and is beong called properly from dataSync
    
    commit 75f647a9
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 14:22:05 2021 +0100
    
        add ecotrack data. rename data dirs
    
    commit ac955cb8
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 13:35:30 2021 +0100
    
        add ecotrack support | fix time bug > see below
    
        fix daylightsavingtimes bug:
        > set timedelta to hours not days
    
        ecotrack support:
        > 4 sensors per unit > merge on unit and habitat and time!
        > sensors log time once every 5 minutes. find closes trackdetection
    
        > calcHabitat() to determine what habitat depending on x,y and PROJ var
        > initTempData() get temp data and store in list of list of tuples
        -> tempData[unit][habitat] returns (time,temp)
        > mergeTemp
    4174320b
    History
    !!!REWRITE HISTORY, SAVE SPACE FOR ECOTRACK, SEE FULL COMMIT MSG FOR DETAILS!!!
    am0ebe authored
    > only full raw data in ecotrack/all/01-raw
    > all other ecotrack/{I,II,III} only link to this
    -> should speed up git and reduce total size and redundancy
    
    > getDates.sh:
    	add earliest latest date for all units
    	write to file
    	add to dataSync
    
    -------------------------------
    All the squashed old commits below:
    -------------------------------
    
    commit baafb4ad (HEAD -> dev, tag: ecotrack-data, tag: ecolux-data, origin/dev, master)
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 18 17:57:32 2021 +0100
    
        recalc *ALL* data + ecotrackimprovement + ... > see below
    
        unknown-tags.sh: fix error message. always call both functions and write to files (tags_info and tags_unknowntags)
        update-gitignore.sh: change name
        dataSync:
          calc mean temperature for all 4 sensors
          filter temp below threshold (cur=12°C)
        datafilter:
          fix format bug where zeros where prepended to species col
          duplicate func make_rel_abs_path (in dataSync and dataFilter) to avoid import error due to circular dependency
    
    commit 84cafbb7
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Wed Mar 17 18:00:05 2021 +0100
    
        fix config. filter: infer proj from filename. add ectrack data I,II,III. start temp: calc mean for
    
    commit 68f4680e
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Tue Mar 16 15:28:27 2021 +0100
    
        continue conf.json and datasync/filter implementation. filter: derive proj for blox from header
    
    commit b419a53b
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Mon Mar 15 14:48:05 2021 +0100
    
        ignore .csv, add ecotrack data, parse conf.json > add blox and project configuration.
    
    commit 99322855
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 18:29:56 2021 +0100
    
        recalc *ALL*. adjust unknowntags script > create tagInfo and unknown-tags automatically from datasync
    
    commit a43a6e59
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 16:01:30 2021 +0100
    
        put blox-configuration into file. unknown-tags.sh takes dir as arg and is beong called properly from dataSync
    
    commit 75f647a9
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 14:22:05 2021 +0100
    
        add ecotrack data. rename data dirs
    
    commit ac955cb8
    Author: am0ebe <am0ebe@gmx.de>
    Date:   Thu Mar 11 13:35:30 2021 +0100
    
        add ecotrack support | fix time bug > see below
    
        fix daylightsavingtimes bug:
        > set timedelta to hours not days
    
        ecotrack support:
        > 4 sensors per unit > merge on unit and habitat and time!
        > sensors log time once every 5 minutes. find closes trackdetection
    
        > calcHabitat() to determine what habitat depending on x,y and PROJ var
        > initTempData() get temp data and store in list of list of tuples
        -> tempData[unit][habitat] returns (time,temp)
        > mergeTemp
Code owners
Assign users and groups as approvers for specific file changes. Learn more.
dataViz.py 1.59 KiB
#!/usr/bin/python3
#
# VIZ (ET/ST)
###################
# sa: ../schrebatrondata/08_alpha_rep1_rep2/02_data_inspect/ana.py
# preq: should work on one huge table with all data!
#	> amount PatchCrossingEvents - Time -> activity histogram per unit/rep
#	> detection_duration_plot
#	> timespan_id_last_seen foreach tag
#	> timespan_id_last_seen foreach group (boxplot)
#	> Graphs per unit:
#		* all
#		* species group A ..
#	> cluster?
#
# CAMTRACK VIZ:
#	>	Total distance of movement
#	>	Minimum and maximum speed
#	>	If possible: turning angle
#
# Processing
##############
# > parse CSV fields neccessary?
# 	movement per tagID: #detections
#	coolOffPhase = 1s?
#	interPatch crossings
# 	most/least movement
# 	tag not detected (dead,malfunct,lost)
# 	... is there a complete tagList somewhere?
# 	sensors with most detection
# 	sensors with no/least detection
# 	need patchConfig.json ?

#from 3k data scripts:
###############

# > collect all tagIDs in list
# > get species - tagID mapping -> xtract from .ods file

# > sort by tagID
# for element in sorted(csvdata, key=attrgetter('tagID')):

# import json
# json.load("patchconfig.json")


speciesDict = {
			"C" 	: "Carabus nemoralis",
			"N" 	: "Nebria brevicollis",
			"PM" 	: "Pterochstichus melanarius",
			"A" 	: "Abax parallelus",
			"P" 	: "Pyrrhocoris apterus",
			"CS" 	: "Coccinella septempunctata",
			"CA" 	: "Cattalus fuscipes",
			"PO" 	: "Poecilus spec.",
			"PR" 	: "Pseudophonus rufipes",
			"HL" 	: "Harpalus latus",
			"H" 	: "Harpalus spec.",
			"PC" 	: "Pterochtichus cristatus",
			"CG" 	: "Carabus granulatus"
			"PQ" 	: "Pterostichus quadrifoveolatus"
}