GIN · internal_only
Family HoldoutMar 4, 2026fc50bf3b244f4af8871cb83dce057891
Description
Hold out simplelocker and wipelocker families entirely from training. Test whether GIN can detect ransomware families it has never seen during training. These two families were chosen as the first holdout test.
Conclusion
Model still achieves 95.7% accuracy and 100% recall on the held-out families. simplelocker and wipelocker appear structurally similar enough to the remaining training families that the model generalises to them. This initially suggested family generalisation might work — but see Experiment 11 for the counterexample.
Test Metrics
Accuracy
95.7%
F1 Macro
95.2%
F1 Malware
96.8%
Precision
93.7%
Recall
100.0%
AUROC
96.0%
Best Val Loss
0.0458
Training Time
962.2000s
Confusion Matrix
| Pred Benign | Pred Malware | |
|---|---|---|
| Actual Benign | 66 | 9 |
| Actual Malware | 0 | 134 |
Configuration
| Hidden Dim | 128 |
| Num Layers | 3 |
| Dropout | 0.5 |
| Batch Size | 4 |
| Learning Rate | 0.001 |
| Weight Decay | 0.0001 |
| Max Epochs | 200 |
| ES Patience | 20 |
| ES Min Epochs | 100 |
| LR Patience | 10 |
| LR Factor | 0.5 |
| Mixed Precision | Yes |
| Random Seed | 42 |
| Epochs Trained | 100 |