How Ideco Measures NGFW Performance Using TRex ASTF and RFC 9411 Methodology

Ideco engineers have published a detailed methodology for benchmarking the performance of their Ideco NGFW Novum product on the Ideco EX hardware platform. The team uses TRex in Advanced Stateful mode to generate realistic L4-L7 traffic and follows RFC 9411 guidelines to ensure reproducible results across different test environments. Key metrics include throughput on UDP 1518-byte frames reaching 200 Gbit/s, TCP/HTTP performance varying from 100 Gbit/s down to 62 Gbit/s depending on object size, and EMIX mixed traffic dropping from 92 Gbit/s with firewall only to 15.5 Gbit/s when all inspection engines are enabled. The methodology incorporates binary search for maximum load, a 1% session drop threshold, and 300-second hold times per test point to capture real-world behavior rather than short-term peaks. All tests were performed without TLS decryption, with separate plans for hardware-based Keysight BreakingPoint testing when decryption is required. The full article includes hardware specifications, traffic profiles, loss calculation formulas, and configuration details to allow anyone to reproduce the measurements.

HabrTranslated from Russian

Read at source

Ideco has shared a comprehensive look at how its engineering team measures the performance of Ideco NGFW Novum on the Ideco EX platform. The article explains why raw performance numbers are meaningless without a clearly documented methodology and describes every parameter used in their lab tests.

Why Performance Numbers Vary Across Tests

The same NGFW device can produce very different results on two different test beds even when both tests are honest. Performance depends on packet size, enabled security functions, number of rules, test duration, and the acceptable loss threshold. Ideco therefore publishes its full methodology so that any reader can reproduce the measurements and understand exactly what each figure represents.

Traffic Generator and Test Modes

The team selected the open-source TRex generator running in ASTF (Advanced Stateful) mode. This choice enables realistic TCP session handling and L7 emulation, which is essential for testing stateful inspection engines. Stateless mode is used only for raw UDP throughput tests with 1518-byte frames. When TLS decryption is enabled on the firewall, the team switches to the hardware-based Keysight BreakingPoint system.

Test Bed Configuration

Both the device under test and the traffic generator use identical server hardware:

  • CPU: Xeon Gold 6338N
  • RAM: 128 GB
  • NICs: Intel E810 2xQSFP28 (100 Gbps)

The topology is a simple two-port setup with TRex port 0 connected to the DUT and TRex port 1 returning the traffic.

Key Metrics and Traffic Profiles

The methodology measures four primary metrics: throughput in Gbit/s, connections per second (CPS), concurrent connections (CC), and latency. Tests are performed with the following profiles:

  • UDP 1518-byte frames (raw L2/L3 ceiling)
  • TCP/HTTP with 64 KB objects
  • TCP/HTTP with 16 KB objects
  • EMIX mixed corporate traffic
  • TCP CPS and TCP CC specific tests

All tests run with a fixed set of firewall rules, IPS signatures, and application control profiles. TLS decryption remains disabled during these runs.

Loss Criterion and Test Duration

Loss is defined as dropped sessions rather than individual packets. The formula uses TRex counters for TCP connection drops, established session drops, and UDP keepalive drops. A test point passes only when session loss stays at or below 1 %. Each load point is held for 300 seconds after a 30-second ramp-up phase to reveal queue buildup, memory pressure, and thermal effects that short bursts would miss.

Results Summary

On the Ideco EX platform the measured maximums are:

  • UDP 1518 B: 200 Gbit/s
  • TCP/HTTP 64 KB: 100 Gbit/s
  • TCP/HTTP 16 KB: 62 Gbit/s
  • EMIX (firewall only): 92 Gbit/s
  • EMIX + IPS: 31 Gbit/s
  • Full NGFW stack: 15.5 Gbit/s
  • TCP CPS: 800 000 new sessions per second
  • TCP CC: 21 000 000 concurrent sessions

The article concludes that performance is always a range that depends on the actual traffic mix and enabled security functions. Full configuration files and commands are provided so the community can replicate the tests.

Meridian digestThe week's most consequential security stories, distilled into one briefing.See the digest

More in Other

Mail.ru Reduces Lifetime 1TB Cloud Storage Promo to Standard 8GB Quota for Select Users
Kimi K3 Open Model Surpasses Closed AI Leaders with Million-Token Context and 896 Experts
Innostage Completes Functional Testing of PT NGFW 1.10.3 Across 156 Criteria