Verifying AI-Written Network Configs Before Deployment

01 /Why generated config needs a different review

A human engineer who does not know a command usually leaves it out. A language model fills the gap with something plausible. The output is well formatted and confidently commented, which is exactly what makes it dangerous: the review instinct that catches a sloppy change does not fire. Treat every generated block as an untrusted patch from a stranger, however good the last one was. This procedure sits in the routing hub because routing policy is where a quiet error travels furthest.

02 /Hazards to look for

Route policy deserves extra suspicion. A prefix-list entry of 10.0.0.0/8 matches only that exact prefix; 10.0.0.0/8 le 24 matches everything inside it up to /24. A route-map sequence with no match clause matches every route. A route-map that ends without a permit sequence denies whatever it did not match, which on a BGP session means withdrawing routes you meant to keep. On a VPN edge a wrong route-target import leaks one customer's routes into another's table.

03 /The verification pipeline

1. 01

   Parse and lint for the exact target

   Run the text through a parser for the platform and software release it will land on, not a generic one. Every line must parse. Then lint for house rules: descriptions present, no clear-text passwords, masks consistent. Unknown commands found here are usually invented ones.

2. 02

   Diff against the running configuration

   Pull the live config and produce a line diff. Read every removed line: generated configs often restate a section and quietly drop entries. Confirm the diff touches only what the change request names.

3. 03

   Check reachability and policy statically

   Load the candidate config into a configuration analysis model and ask concrete questions: can the management subnet still reach every device, does the ACL still permit monitoring, which prefixes does the route-map now accept. Offline analysis finds ordering and implicit-deny errors without touching a device.

4. 04

   Test in a lab or emulator

   Apply the change to a lab device or emulator running the same release, with neighbours that mimic production. Watch adjacencies form, routes arrive and traffic pass. A first-hop redundancy pair should be failed over during the test, not just configured.

5. 05

   Stage the rollout with a rollback timer

   Before applying, arm an automatic way back: a timed reload that restores the saved configuration, or a confirmed commit that reverts unless confirmed within minutes. Apply to one device, confirm, then the next. Do not save until the change is proven.

6. 06

   Verify with show commands, then confirm

   Compare live state to the stated intent: interface state, neighbours, route counts, ACL hit counters, a test flow. Only then cancel the timer or confirm the commit, and save.

! save the known-good config first
write memory
! arm the safety net: restore saved config in 10 minutes
reload in 10
configure terminal
 ! paste the reviewed, diffed change here
end
show ip route summary
show ip bgp summary
show access-list 110
! intent confirmed: cancel the timer, then save
reload cancel
write memory

04 /Worked example: verify an AI config for a new BGP peer

The request was ordinary: add an external BGP neighbour at 192.0.2.9 in AS 64500, accept only 198.51.100.0/24 and its subnets from it, and announce the local aggregate. The assistant returned twelve tidy lines with comments. Three of them were wrong in ways that read well, which is the pattern this procedure exists to catch. The transcript below shows what the diff and the parser said before anything touched the router.

$ diff -u running.cfg candidate.cfg
+ip prefix-list FROM-64500 seq 5 permit 198.51.100.0/24
+route-map IN-64500 permit 10
+ match ip address prefix-list FROM-64500
+router bgp 64496
+ neighbor 192.0.2.9 remote-as 64500
+ neighbor 192.0.2.9 route-map IN-64500 in
+ neighbor 192.0.2.9 soft-reconfiguration inbound always
- neighbor 192.0.2.5 description transit-b
$ conflint --platform ironware --release 8.0 candidate.cfg
candidate.cfg:7: unknown keyword 'always' after 'soft-reconfiguration inbound'
candidate.cfg:1: prefix-list permits exact /24 only; request said 'and its subnets'
candidate.cfg:8: removal of existing neighbor description (not in change request)

Line one is the quiet error. permit 198.51.100.0/24 matches that prefix and nothing longer; the request wanted le 32 or at least le 28. The invented keyword on line seven would have been rejected at paste time, which is the harmless failure. The removed description on line eight is the one nobody asked for: the model restated the BGP section and dropped a line it did not understand. The diff caught it; a reviewer reading only the added lines would not have.

Keep the artefacts with the change record, and keep the device syslog that shows the commit alongside them. The same discipline that makes edge logs usable as evidence, accurate timestamps, untranslated addresses and no gaps, applies to the log of who changed a router and when.

05 /Platform and release syntax

Models trained on public text have seen far more of the market leaders' syntax than of anything else, and they blend it. A request for a switch trunk may come back with keywords from a different family, or with a command from a release three years newer than the one installed. Name the platform, the software train and the exact release in the request, and check the result against that release's own help output. For older gear the problem is worse; checking generated answers about end-of-life IronWare shows the pattern on one platform, and the IronWare command reference gives the native forms.

06 /Avoiding management lockout

The costliest generated mistake is the one that cuts off the person applying it: an ACL on the vty lines that omits the admin subnet, a management VLAN renumbered, SSH disabled before its replacement is tested, AAA pointed at a server that is not reachable. Keep an out-of-band path, a console server or a separate management network, and never apply management-plane changes without a rollback timer. On a core router the out-of-band path is not optional.

07 /Expert tips for reviewing generated configs

• Ask for the show commands first. Request the verification commands and expected output before the configuration. A model that cannot say what success looks like has not understood the change.
• Search the output for defaults you did not request. Timers, authentication, administrative distance and MTU appear in generated blocks because they appeared in training text. Every unrequested value is a change to justify or delete.
• Diff both directions. Read removed lines as carefully as added ones. Restated sections drop entries.
• Test the failure, not just the success. Shut the new neighbour, pull the route, fail the redundancy pair. A change that survives its own failure mode is verified; one that only works is merely present.
• Keep the prompt with the ticket. When the same class of error recurs, the prompt is where the fix goes, and the record shows which wording produced which mistake.
• Never let the assistant apply anything. Generation and deployment are separate steps with a person between them, whatever the tooling offers.

Two companion procedures narrow the checks: recomputing generated subnet and wildcard math and a review order for tool-drafted changes.