well, obviously, you need to keep track of the faces you’ve already seen before 
one way to do this, would be to use face recognition, so you only count faces, that havent been seen before …
which means, you need to setup a database of face features to track between frames !
this would be a good starting point
ps: rather ditch dlib.get_frontal_face_detector(), it’s terrible slow !