Code owners
Assign users and groups as approvers for specific file changes. Learn more.
notes 9.96 KiB
v## SW from tboy old workstation
https://ctrax.sourceforge.net/
walking fly tracker
HOBOWare Lite
temperature logger
StreamPix5
realtime digital video recording to PC memory or hdd
batch processor
https://www.norpix.com/products/streampix/streampix.php
Matlab (.m files)
any2ufmf
fviewflytrax
avidemux
yEd - graph editor - .graphml files
batchprocessor
process sequence files to avi
ssdcleanup
###############################
## Emilio
###############################
> !! if beetle runs in circle it is in "escape mode" which doesnt represent "natural" behaviour (explore, quick move from a to b)
-> think about different setup. Bigger Arena?
> fps can be low. Try 3!
Depends on movement speed and estimated error
calc error by comparing distance of 40fps with 3fps
> duration of video can be short. eg 10 minutes.
-> no need for many fridges
###############################
## Software A - Frame grab
##############################
# setup Vimba
> see Vimba_installation_under_Linux.pdf
> download and unpack Vimba SDK
> tar -xzf ./Vimba.tgz
> sudo ./VimbaGigETL/Install.sh
## SETUP GT1920C:
> connect lap and cam directly via ethernet
> set MTU to 8228 (jumbo frames/packets)
> set fixed IP for eth-adapter first and then cam (use Vimbaviewer -> force IP)
eth 169.254.100.1
cam 169.254.x.x # on restart will pick random IP...
mac
000f310338D3
000f310338D4
sub 255.255.0.0
(gat 0.0.0.0 local / none)
-> ip address of each adapter needs to be on a unique subnet
-> for multiple cams calc bandwith and use switch
# What bandwidth do i have? Do i need multiple ports?
bandwith = fps * pixel format(bpp) * resolution (* ncams)
StreamBytesPerSecond = 17 * 3 * 1936 * 1216 * 1 = 8 456 448 ~= 8,4 MBps < 125MBps
1. Determine max_speed with highest fps!
2. Take max_speed and reduce fps so it still fits 2*max_speed
-> subsample same video ?
3. calc mean_err through comparing 1. and 2. -> add to 2. as it will be lower.
!! exposure and fps: on 10fps exposure can't be more then 100ms!
!! exposure and fps: on 17fps exposure can't be more then 59ms!
Best practice: set gain to lowest possible val and increase exposure as needed
!! if you use more than one cam on one interface, the available bandwidth has to be shared between interfaces.
################################ bandwidth calculations for Alvium G1 240C
MAX BANDWITH for Gigabit ethernet port ~= 125 Mbps
max res: 1936x1216
max fps: 49
pixelformat: RGB8 > different options. eg 10bit/12bit RGB/MONO.
1936 * 1216 * 49 * 1 ~= 115,3 MBps >> MONO8 @ maxFPS
1936 * 1216 * 17 * 1 ~= 40,0 MBps >> MONO8 @ 17FPS
@ 1 Min -> 2.4GB
@ 30 Mins -> 144GB
with 60 recordings: 8.6TB
1936 * 1216 * 17 * 3 ~= 120,1 MBps >> RGB8 @ 17FPS
@ 1 Min -> 7.2GB
@ 30 Mins -> 216.1GB
with 60 recordings: 13 TB
1936 * 1216 * 10 * 3 ~= 70,6 MBps
@ 1 Min -> 4.2 GB
@ 30 Mins -> 127.1 GB
with 60 recordings: 7.6TB
Storage:
ws1:
2TB: 2*1TB (7200RPM)
8Gb DDR3 RAM
Intel Xeon 3.6Ghz (4 cores)
ws2: 4TB: 3 (7200RPM = 6GB/s!) + 1
8GB DDR3 RAM
default bandwidth:115MBps
Save "reasonable" settings in XML.
If dark, set ExposureTimeAbs
Q: how to set fps to 3?
#####################################
## Background Substraction - BGS
####################################
#define bgs_register(x) static BGS_Register<x> register_##x(quote(x))
## >> glue together macro?
quote() adds ""
bgs_register(Tapter)
>> static BGS_Register<Tapter> register_Tapter("Tapter")
/usr/include/opencv4/opencv2/opencv.hpp
PCA?
> difference
virtual();
virtual(){ /*empty*/ }
virtual() = 0; #pure virtual
> also difference
> virtual dtor(): if pointer to base-class deletes object
> pure virtual dtor(): need to also define function body, cuz dtor is special function whis is not overriden
> interface class (needs pure virtual??)
> abc - abstract base class. can't be instantiated anymore
> abc <> interface?
// IplImage is oldskool mat and not supported anymore..
> use smartpointer like so:
auto videoAnalysis = std::make_unique<VideoAnalysis>();
videoCapture = std::make_unique<VideoCapture>();
frameProcessor = std::make_shared<FrameProcessor>();
###########
libarchive stuff
archive_read_xxx()
archive_write_xxx()
struct archive_entry
huge workaround for corrupted files
clock
random
command arg parser
???
#if CV_MAJOR_VERSION > 3 || (CV_MAJOR_VERSION == 3 && CV_SUBMINOR_VERSION >= 9)
IplImage _frame = cvIplImage(img_input);
frame = &_frame;
#else
frame = new IplImage(img_input);
#endif
#smartpointer??
#elif CV_MAJOR_VERSION >= 3
cv::Ptr<cv::BackgroundSubtractorMOG2> mog;
#endif
> What Tboy do... why did he do it?
> forked branch bgslib_qtgui_2.0.0
> Tapter
> adapter for model
> was probably copied from LBAdaptiveSOM.cpp
--> which is disabled in openCV4
Which Background Subtraction Algo to use??
median, mean, framedifference
+ simple, fast,
- not robust if light/bg changes
- slow changes ?
> adaptive bg?
fuzzy?
mixture
NAMES
Kernel?
LBSP?
Multilayer?
Sigma-Delta?
Vibe, T2F ,dp ,lb ,vumeter?
choquet, sugeno, zivkovic, pratimediod, LOBSTER
Test/Use most common >> Ground Truth
Frame Difference
WeightedMovingMean / Variance
LBAdaptiveSOM
MOG2 (Mixture Of Gaussian) MixtureOfGaussianV2.h
KNN (K Nearest Neighbour)
> fast for small fg obj
> TRY!
FuzzySugenoIntegral.h
LSBP - Local Binary Similarity Patterns - (2013)
LSBP-based GSoC ?
SuBSENSE: improved spatiotemporal LBSP + color features (2014)
Combineable with
ROI
Canny Edge Detection
> bg modeling to update BG (eg moving trees) > pixel with threshold
> optic flow (camera is also moving. ) > vectoral estimation of own movement
features
edge
canny edge detector + calc contour
> https://en.wikipedia.org/wiki/Canny_edge_detector
roi
crop
color
RGB - not so robust by itself (sensitive to illumination, shadows, oscillations ...)
YUV
YCrCb - brightness, chroma, color
texture
robust to illumination and shadow
eg Local Binary Pattern (LBP)
https://github.com/murari023/awesome-background-subtraction (2021 new stuff!)
https://learnopencv.com/background-subtraction-with-opencv-and-bgs-libraries/
http://docs.opencv.org/2.4/doc/tutorials/imgproc/gausian_median_blur_bilateral_filter/gausian_median_blur_bilateral_filter.html
https://hackthedeveloper.com/background-subtraction-opencv-python/ #mog2 + knn in python
https://docs.opencv.org/4.3.0/d4/dd5/classcv_1_1bgsegm_1_1BackgroundSubtractorGSOC.html#details #GSOC LSBP ALGO from openCV bgsegm.hpp
https://openaccess.thecvf.com/content_cvpr_workshops_2014/W12/papers/St-Charles_Flexible_Background_Subtraction_2014_CVPR_paper.pdf
https://www.scitepress.org/Papers/2018/66296/66296.pdf #vehicle tracking latvia. 2018, BackgroundSubtractorMOG, BackgroundSubtractorMOG2 (zivkovic)
https://www-sop.inria.fr/members/Francois.Bremond/Postscript/AnhTuanAVSS14.pdf 2014
https://arxiv.org/pdf/1803.07985.pdf # visual animal tracking (2018)
https://arxiv.org/pdf/1507.06821.pdf # Multimodal Deep Learning for Robust RGB-D Object Recognition (2015)
https://towardsdatascience.com/background-removal-with-deep-learning-c4f2104b3157?gi=2ef3a5272e5d (2017 Background removal with deep learning)
https://opencv.org/courses/ #xpensive ai course
https://www.fast.ai/ #free ai course
Build with Python or C++?
##################
# computervision #
##################
background subtraction (bgs)
segmentation
Semantic Segmentation (ai)
detection (feature, object)
classification (category recognition)
Challenges: Occlusion, (Sensor-)Noise, changing external conditions( lighting, Shadows, fog, reflection )
> pre-training if lack of data
> corrupt data to guarantee robust learning
######
# AI #
######
Convolutional Neural Networks (CNNs)
https://www.researchgate.net/publication/337401161_Fundamental_Concepts_of_Convolutional_Neural_Network (2020 37p)
> Kernel
> filter?
> preprocessing > Training > Parameter init > regularization > optimize
RGB-D - RGB + depth >> eg Kinect-Sensors
KNN - k-nearest neighbors algorithm
Monte Carlo tree search algorithm (alphaGo...)
RGB-D descriptor that relies on a K-Means
HMP - hierarchical matching pursuit ( hierarchical sparse-coding method; learn features from multiple channel input)
encoding depth information into three channels (HHA encoding) feature hierachy
cascade of Random Forest classifiers that are fused in a hierarchical manner ?? #Random Forest
metrics:
recall - how many relevant items are retrieved?
precision - how many retrieved items are relevant?
> https://en.wikipedia.org/wiki/Recall_(information_retrieval)
?
automatically learn a fusion strategy for the recognition
task – in contrast to simply training a linear classifier on
top of features extracted from both modalities
MISRE - Multiple Input Structures with Robust Estimator
gradient-based learning
> MNIST supervised image classification
MNIST - large db of hand-written digits (used for ML and ImagePRocessing)
> also see classifiers: https://en.wikipedia.org/wiki/MNIST_database#Classifiers
multimodal learning
the parameters of all layers were adapted using a fixed learning rate schedule
At test time the task of the CNN is to assign the correct class label to a previously unseen object instance.
Tasks:
Classification, clustering, regression, Summarization, Categorization, Natural language processing,
##################################
# STAT
##################################
Regression
Hadamard Product
Bernoulli Distribution
Discrete Uniform Distribution
##################################
# CMAKE
##################################
sublime: install package CMAKE (syntax hilight)
apt install libarchive-dev python3-pybind11
##################################
init(img_input, img_output, img_bgmodel);
#if CV_MAJOR_VERSION > 3 || (CV_MAJOR_VERSION == 3 && CV_SUBMINOR_VERSION >= 9)
IplImage _frame = cvIplImage(img_input);
frame = &_frame;
#else
frame = new IplImage(img_input);
#endif
## hardware tower old
(opt/later) setup tower1 (defunct again!) as cheap testing machine!
> get NIC working
> give name connect inet
--> as terminal to connect to server
--> later: use for DMX lighting