Back to Help Index

ML/AI Insights Workbench

VersionGopher turns scan evidence into feature vectors, then lets analysts explore what is common, what is rare, what belongs together, and what looks wrong. The workbench is not a magic verdict engine. It is a way to make millions of file, package, hash, path, signature, and vulnerability observations navigable.

A dot is one scanned file observation: one file from one scan, identified by scan id and scanned file id, represented as a numeric evidence vector.

Discovery

DBSCAN

DBSCAN asks: which files are outside dense neighborhoods? It is best for finding needles, odd paths, one-off signer behavior, strange co-location, and files that do not belong with nearby evidence.

Discovery

K-Means

K-Means asks: what cohorts exist in this evidence? It groups files into K clusters, then shows cluster sizes, centroids, distinguishing features, and distance-to-centroid examples.

Prediction

Logistic Regression

Logistic Regression uses labeled VG data to predict probability. It is useful when you need explainable feature weights, threshold tuning, precision, recall, false positives, false negatives, and ROC/AUC.

Prediction

Decision Tree

Decision Tree shows rule paths in plain terms: if these features look this way, the model lands in this leaf. It is useful for feature importance, leaf probability, and reviewable rules.

Map

PCA-3D Projection

PCA-3D compresses the full normalized feature vector into three visual components. The model still runs on the full feature matrix; the 3D map is a lossy view that helps your eyes inspect structure.

Evidence

Model vs Overlay

Model explanation says which features made the file odd or grouped it. Overlay context adds CVE, KEV, malware hash, package advisory, or known-bad catalog evidence after the model result.

What The Main Controls Do

Control	Plain-English meaning	When to change it
Preset	A reviewed starting point chosen from large forensic discovery sweeps.	Change it when you want a different discovery lane, such as ELF path/package, PE path/signature, co-location, package/dependency, or hash identity.
Mode	Discovery / Clustering is unlabeled exploration. Prediction / Classification uses labeled examples.	Use discovery when you do not know what is hiding. Use classification when you have labels and want predicted risk.
Algorithm	DBSCAN finds dense neighborhoods and noise. K-Means forms K cohorts. Supervised models predict labels.	Switch algorithms when the question changes from "what is isolated?" to "what groups exist?" or "what outcome is likely?"
Projection	PCA-3D uses the full vector for the map. Manual feature axes plot selected fields directly.	Stay on PCA-3D for structure. Use manual axes to inspect a specific feature, especially after checking whether it is continuous or binary-like.
Scope	The scan population being queried, such as all visible scans or one selected scan.	Narrow scope when a broad organization view hides site-specific oddities or takes too long.
Rows	How many findings are shown in the rail. It does not change how the model scores the sampled evidence.	Increase it when you want a longer analyst queue; decrease it when triaging quickly.
Feature source and family	The evidence lane used to build vectors: file/binary forensics, path anomalies, PE, ELF, signature/cert, hash identity, co-location, package/dependency, malware-adjacent, or CVE context.	Choose the family that matches the question. CVE and KEV are overlays unless you intentionally select vulnerability context.

Knobs, Buttons, And Wheels

DBSCAN tuning

eps is the neighborhood radius. Lower values make neighborhoods tighter and usually create more outliers.
min points is how many nearby files are needed before a neighborhood counts as dense.
distance metric changes how similarity is measured: cosine for marker mix, euclidean for shape and magnitude, manhattan for additive gaps.
rare prevalence asks how uncommon a marker must be before VG treats it as review-worthy.

K-Means tuning

K is the number of cohorts to form.
scaler changes how raw feature ranges are normalized before distance calculations.
weight profile emphasizes different forensic lanes, such as PE-heavy, ELF-heavy, or co-location-heavy discovery.
Silhouette and elbow sweeps help decide whether K is separating real structure or forcing arbitrary groups.

3D projection controls

Drag to rotate the 3D map.
Use the mouse wheel to zoom.
Shift-drag or the plot controls can pan the view.
Click a point to open the evidence drawer for that exact scan/file.
Click a finding or cluster to highlight matching points and filter the queue.

Buttons and selectors

Apply tuning: reruns the selected model settings.
Refresh: reloads the current scope from the server.
Export JSON: saves the current model result and evidence references for review.
Feature details: opens the feature manifest, histograms, axis eligibility, and formulas.
Color, size, shape: change overlays on the 3D map without changing the model.

Hard Hitting Example

ProgramData DLL With Malware Overlay

Imagine the rail highlights softokn3.dll under C:\ProgramData\.... A DLL name that normally belongs to a known software stack is now sitting in a broad writable location. The file is not just "a CVE row"; it is one scanned file observation with a path, hash, filename, signature evidence, nearby files, package context, and optional CVE or malware overlays.

DBSCANNoise point outside dense path/signature neighborhoods.

K-MeansFar from the centroid for ordinary signed application DLL cohorts.

Model explanationPath depth, filename entropy, writable directory, unsigned or odd signer evidence, and unusual neighbors.

Overlay contextMalware hash hit, CVE/KEV context, or package advisory if those signals exist.

The right analyst action is direct and concrete: open the exact scan/file, verify the path and signer, compare the hash to known-good inventory, inspect neighboring scripts or archives, isolate the host if the malware overlay is credible, and rescan after remediation.

The punchline is not "AI says bad." The punchline is "VG found a file whose forensic feature vector does not behave like the rest of the fleet, and the evidence drawer gives you the exact scan/file to prove or dismiss it."

What Is New In 0.7.6

A cockpit, not a report

Findings, the 3D map, and the evidence drawer are linked so a selected file stays visible from model result to raw scan evidence.

Forensic-first defaults

The default views start from file, path, binary, signer, hash, co-location, package, PE, and ELF evidence. CVE and KEV context are overlays unless you intentionally choose vulnerability context.

Richer PE features

PE section counts, entropy, suspicious section names, overlay size, import/export shape, and Rich Header/toolchain clues can contribute to discovery and show up in explanations.

Analyst tuning

DBSCAN radius and density, K-Means K, scaling, distance rules, feature family, and projection mode remain adjustable because each customer's data has its own shape.

How To Read A Finding

What VG found

A grouped finding should name the file, package, hash identity, path family, cluster, or cohort. Repeated CVE rows should not appear as separate findings when they are the same underlying file signal.

Why it matters

This is the security sentence: writable path, unexpected signer, known-good name with unknown hash, known-good hash in a strange path, malware-adjacent neighbor, rare package dependency, or isolated ELF/PE structure.

Which algorithm found it

DBSCAN outlier, DBSCAN cluster, K-Means cohort, Logistic Regression probability, or Decision Tree rule path should be explicit.

What to do next

Open the exact scan/file, review the evidence drawer, compare similar files, confirm whether the behavior is expected in that organization, and record analyst labels when the workbench asks you to teach VG.

Feature Sources You Will See

File / Binary Forensics Path and Filename Anomalies PE Header Forensics ELF Header Forensics Signature / Certificate Forensics Hash / Identity Mismatches Co-location Anomalies Malware-Adjacent Files Package / Dependency Anomalies CVE / Vulnerability Context

Continuous features can be plotted as axes. Binary flags are better as colors, shapes, filters, or tooltip context. Categorical fields such as extension, signer, package manager, or architecture are model inputs only after they are encoded or used as filters.

Validation Signals

Clustering validation

K-Means uses silhouette and elbow checks to see whether clusters are separating meaningful structure. DBSCAN uses cluster count, outlier count, core/border/noise roles, and analyst review of representative points.

Classification validation

Logistic Regression and Decision Tree views should show threshold, confusion matrix, precision, recall, false positives, false negatives, and ROC/AUC where labeled validation data exists.

Defaults are seeds, not truth. The current presets come from large reviewed discovery sweeps across VersionGopher field data. Your data can have a different shape, so the workbench keeps the parameters adjustable.

Common Misreads To Avoid

A dot is not a whole scan. It is one scanned file observation in the selected scope.
A dot is not a CVE row. CVE and KEV can be overlays, but forensic discovery should work without them.
Binary features make rails. If a manual axis is mostly zero or one, the chart will naturally collapse into two bands.
PCA-3D is a map, not the model. It preserves enough structure to navigate, but some distance is lost during projection.
Malware overlay is context. It strengthens urgency, but the model explanation should still show the forensic reasons.
Labels can leak. Supervised models must avoid training on fields that simply restate the answer.

Fast Workflow

Start with a reviewed preset that matches your question.
Stay in PCA-3D and scan for isolated points, dense cohorts, and far-from-centroid examples.
Click a finding, point, or cluster to keep the rail, map, and evidence drawer linked.
Read model explanation first, then overlay context.
Open the exact scan/file only when the evidence looks worth analyst time.
Adjust eps, min points, K, scaler, feature family, or scope when the current view is too noisy or too bland.