bubbletea0.1.0-SNAPSHOTThis is a basic example for working in Clojure with JavaFX, OpenCV and simple plotting dependencies
| (this space intentionally left almost blank) | ||||||||||||||||||||||||
JavaFX is used through the fn-fx library. When the application is run through thi.ng/geom-viz is for making plots and other pretty SVG things OpenCV is used through origami library which comes with precompiled libs/jars for common systems (linux/windows/mac) so we inject those into our project with FranzXaver is for converting the output SVGs to something that can be put into a Group in JavaFX. | (ns buubletea.core
(:require [fn-fx.fx-dom :as dom] ;; The JavaFX libraries
[fn-fx.diff :refer [component defui render should-update?]]
[fn-fx.controls :as ui]
[thi.ng.geom.core :as g] ;; The graphing libraires
[thi.ng.geom.viz.core :as viz]
[thi.ng.geom.svg.core :as svgthing]
[thi.ng.math.core :as m :refer [PI TWO_PI]]
[opencv3.core :as oc] ;; The OpenCV libraries
[opencv3.video :as ov]
[opencv3.utils :as ou])
(:import [afester.javafx.svg SvgLoader])) | ||||||||||||||||||||||||
Globals | |||||||||||||||||||||||||
(def main-font (ui/font :family "Helvetica" :size 20))
(def default-webcam-options ;; this is used in (-main) and as a default when using 'webcam-image' with no arguements
{:frame
{:color "00" :title "video"}
:video
{:device 0
:width 320
:height 240
:datatype oc/CV_8UC3 }}) | |||||||||||||||||||||||||
Event Handler | |||||||||||||||||||||||||
This is the event handler multimethod through which all events go through. It will 'switch' on the :event key | (defmulti handle-event
(fn [state event]
(:event event))) | ||||||||||||||||||||||||
Get an image from the webcam | |||||||||||||||||||||||||
Get a picture from the webcam through OpenCV and return an OpenCV Mat (color is in RGB order) | (defn webcam-image
([]
(webcam-image default-webcam-options))
([webcam-options]
(let [webcam-capture (ov/new-videocapture)
webcam-mat (oc/new-mat)]
(doto webcam-capture
(.open (int (-> webcam-options :video :device)))
(.set ov/CAP_PROP_FRAME_WIDTH (-> webcam-options :video :width))
(.set ov/CAP_PROP_FRAME_HEIGHT (-> webcam-options :video :height)))
(ou/.read webcam-capture webcam-mat)
(oc/cvt-color! webcam-mat oc/COLOR_BGR2RGB)
(.release webcam-capture)
webcam-mat))) | ||||||||||||||||||||||||
Go into an OpenCV Mat and return the internal byte-array buffer. We do it by making an empty byte buffer and copying over the Mat data | (defn get-mat-byte-buffer
([mat]
(get-mat-byte-buffer mat (.width mat) (.height mat) (.channels mat)))
([mat width height channels]
(get-mat-byte-buffer mat (* width height) channels))
([mat number-of-pixels channels]
(get-mat-byte-buffer mat (* number-of-pixels channels)))
([mat size]
(let [image-buffer (byte-array size)]
(.get mat 0 0 image-buffer)
image-buffer))) | ||||||||||||||||||||||||
Take in a byte array of RGB value (RGBRGBRGB..) and some dimensions and turn it into a JFXImage. JavaFX has a limited number of options other than RGB, so it's hardcoded for now to 3-chan RGB. see: javafx.scene.image.PixelFormat | (defn make-jfx-image-from-byte-buffer
[image-buffer width height]
(let [writable-image (new javafx.scene.image.WritableImage width height)
^javafx.scene.image.PixelWriter pixel-writer (.getPixelWriter writable-image)]
(.setPixels pixel-writer 0 0 width height
(javafx.scene.image.PixelFormat/getByteRgbInstance)
image-buffer
0
(* 3 width))
writable-image)) | ||||||||||||||||||||||||
this is a simple wrapper around the last function.. but sometimes u want to reuse the buffer so it's good to hve both | |||||||||||||||||||||||||
Take an OpenCV Mat and turn it into a JFXImage | (defn make-jfx-image-from-mat
([mat]
(make-jfx-image-from-byte-buffer
(get-mat-byte-buffer mat)
(.width mat)
(.height mat)))) | ||||||||||||||||||||||||
Build Histogram from Image | |||||||||||||||||||||||||
Takes a byte which will show as signed number over the range -126:126 and turns it into an int on the 0:255 range | (defn unsigned-byte-to-int
[ubyte]
(cond
(neg? ubyte) (int (+ 256 ubyte)) ;; negative numbers have the sign bit flipped and are inverted
:else (int ubyte))) | ||||||||||||||||||||||||
Takes a SEQUENCE of repeating continuous values and bins them it into a vector based on their values. Can be given a MAX-INTEGER value to initialize the continuous bin vector Otherwise looks for the max-value before bining | (defn build-histogram-vector
([input-sequence]
(build-histogram-vector input-sequence (apply max input-sequence)))
([input-sequence max-integer]
(reduce
(fn [old-histogram-vector new-value]
(let [new-int (unsigned-byte-to-int new-value)]
(assoc
old-histogram-vector
new-int
(inc (get old-histogram-vector new-int)))))
(vec (repeat max-integer 0))
input-sequence))) | ||||||||||||||||||||||||
Take a vector of numbers [a b c d ..] and makes an indexed-pair version [[0 a] [1 b] [2 c] [3 d] ..] | (defn index-vector ([vector] (index-vector vector (count vector))) ([vector length] (map (fn [i] [i (get vector i)]) (range 0 (dec length))))) | ||||||||||||||||||||||||
Takes a string and turns it into an input stream | (defn string->stream
([s] (string->stream s "UTF-8"))
([s encoding]
(-> s
(.getBytes encoding)
(java.io.ByteArrayInputStream.)))) | ||||||||||||||||||||||||
Use the FranzXaver library to turn a string of XML describing an SVG into a JavaFX compatible Group Node (which shows up as a picture) This is using Batik under the hood somehow | (defn svg-to-javafx-group [svg-xml-string] (.loadSvg (SvgLoader.) (string->stream svg-xml-string))) | ||||||||||||||||||||||||
Generate a histogram spec of the given size | (defn histogram-spec
[width height max-value min-value]
{:x-axis (viz/linear-axis
{:domain [0 255]
:range [0 width]
:major 1
:pos height ;280
:label (viz/default-svg-label int)
:visible false})
:y-axis (viz/linear-axis
{:domain [0 max-value]
:range [height 0]
:major 10
:minor 5
:pos 0
:visible false }) ;; based off the example in geom-viz
:grid {:minor-y true}}) | ||||||||||||||||||||||||
Generate a spec for the data in the histrogram | (defn bar-spec
[num width histogram-values]
(fn [idx col]
{:values histogram-values
:attribs {:stroke col
:stroke-width (str (dec width) "px")}
:layout viz/svg-bar-plot
:interleave num
:bar-width width
:offset idx})) | ||||||||||||||||||||||||
Given an image's byte buffer and it's length, returns a histogram of a given same size. The histogram is generated using thi.ng/geom-viz and then transfromed into a JavaFX Group | (defn image-to-histogram
[image-byte-vector length output-width output-height]
(let [ histogram-vector (build-histogram-vector image-byte-vector length)
max-value (apply max histogram-vector)
min-value (apply min histogram-vector)
indexed-histrogram-vector (index-vector histogram-vector length)]
(svg-to-javafx-group (-> (histogram-spec output-width output-height max-value min-value)
(assoc :data [((bar-spec 1 2 indexed-histrogram-vector) 0 "#000")])
(viz/svg-plot2d-cartesian)
(#(svgthing/svg {:width output-width :height output-height} %))
(svgthing/serialize))))) | ||||||||||||||||||||||||
Image EntryThis holds one image and its accompanying analysis | |||||||||||||||||||||||||
(defui ImageEntry
(render [this {:keys [done? idx text image histogram image-edges]}]
(ui/border-pane
:padding (ui/insets
:top 10
:bottom 10
:left 0
:right 0)
:left (ui/check-box
:font main-font
:text text
:selected done?
:on-action {:event :swap-status :idx idx})
:right (ui/button :text "X"
:on-action {:event :delete-item :idx idx})
:bottom (ui/h-box
:children [(ui/label :graphic (ui/image-view :image image))
histogram
(ui/label :graphic (ui/image-view :image image-edges))
#_(ui/button :text "X"
:on-action {:event :delete-item :idx idx})])))) | |||||||||||||||||||||||||
MainWindowthe root node of the scene-graph. It will track the scene size/resizing | |||||||||||||||||||||||||
(defui MainWindow
(render [this {:keys [todos]}]
(ui/v-box
:style
"-fx-base: rgb(255, 255, 255);
-fx-focus-color: transparent;"
:padding (ui/insets
:top-right-bottom-left 25)
:children [(ui/text-field
:id ::new-item
:prompt-text "What needs to be done?"
:font main-font
:on-action {:event :add-item
:fn-fx/include {::new-item #{:text}}})
(ui/scroll-pane
:vbar-policy javafx.scene.control.ScrollPane$ScrollBarPolicy/ALWAYS
:fit-to-height false
:fit-to-width false
:content
(ui/v-box
:children
(map-indexed
(fn [idx todo]
(image-entry (assoc todo :idx idx)))
todos)))]))) | |||||||||||||||||||||||||
Stagethe JavaFX top level container that stands for a window The stage has a scene container for all content ie. a scene-graph of nodes. Each Stage/Window displays one scene at a time | |||||||||||||||||||||||||
(defui Stage
(render [this args]
(ui/stage
:title "ToDos"
:min-height 600
:shown true
:scene (ui/scene
:root (main-window args))))) | |||||||||||||||||||||||||
(defmethod handle-event :swap-status
[state {:keys [idx]}]
(update-in state [:todos idx :done?] (fn [x]
(not x)))) | |||||||||||||||||||||||||
(defmethod handle-event :delete-item
[state {:keys [idx]}]
(update-in state [:todos] (fn [itms]
(println itms idx)
(vec (concat (take idx itms)
(drop (inc idx) itms)))))) | |||||||||||||||||||||||||
(defmethod handle-event :add-item
[state {:keys [fn-fx/includes webcam-params]}]
(println (:current-webcam-params state))
(let [webcam-params (:current-webcam-params state)
width (-> webcam-params :video :width)
height (-> webcam-params :video :height)
snapshot (webcam-image webcam-params)
snapshot-buffer (get-mat-byte-buffer snapshot)
snapshot-edges (-> snapshot
(oc/cvt-color! oc/COLOR_RGB2GRAY)
(oc/canny! 300.0 100.0 3 true)
(oc/bitwise-not!)
(oc/cvt-color! oc/COLOR_GRAY2RGB))]
(update-in state [:todos] conj {:done? false
:text (get-in includes [::new-item :text])
:webcam-params webcam-params
:image (make-jfx-image-from-byte-buffer snapshot-buffer width height)
:histogram (image-to-histogram snapshot-buffer 256 width height)
:image-edges (make-jfx-image-from-mat snapshot-edges)}))) | |||||||||||||||||||||||||
(defmethod handle-event :toggle-text [state event] (assoc state :comment-text "hello")) | |||||||||||||||||||||||||
(defmethod handle-event :default [state event] (println "No hander for event " (:type event) event) state) | |||||||||||||||||||||||||
Launching fn-fx
| |||||||||||||||||||||||||
This is where we initialize the whole fn-fx monster | (defn -main
[]
(let [starting-webcam-params default-webcam-options
width (-> starting-webcam-params :video :width)
height (-> starting-webcam-params :video :height)
first-image (webcam-image starting-webcam-params)
first-image-buffer (get-mat-byte-buffer first-image)
first-image-edges (-> first-image ; THIS CHANGES THE MAT INPLACE!
(oc/cvt-color! oc/COLOR_RGB2GRAY)
(oc/canny! 300.0 100.0 3 true)
(oc/bitwise-not!)
(oc/cvt-color! oc/COLOR_GRAY2RGB))
data-state (atom {:current-webcam-params starting-webcam-params
:todos [{:done? false
:text "A confused face when the app launches..."
:webcam-params starting-webcam-params
:image (make-jfx-image-from-byte-buffer first-image-buffer width height)
:histogram (image-to-histogram first-image-buffer 256 width height)
:image-edges (make-jfx-image-from-mat first-image-edges)}]})
handler-fn (fn [event]
(try
(swap! data-state handle-event event)
(catch Throwable ex
(println ex))))
ui-state (agent (dom/app (stage @data-state) handler-fn))]
(add-watch data-state :ui (fn [_ _ _ _]
(send ui-state
(fn [old-ui]
(try
(dom/update-app old-ui (stage @data-state))
(catch Throwable ex
(println ex))))))))) | ||||||||||||||||||||||||
(comment (-main) ) | |||||||||||||||||||||||||