feat: add multi-interface support for parallel packet sending#82
Open
feat: add multi-interface support for parallel packet sending#82
Conversation
- Add EnumStream(ctx, domain, callback func(Result)) error - Add VerifyStream(ctx, domains, callback func(Result)) error - Introduce streamCollector (implements outputter.Output) that forwards each result to the callback in real-time, no buffering - Refactor: extract buildOptions(), buildDictChan(), parseResult() to eliminate duplication between blocking and stream paths - Remove obsolete Timeout/DynamicTimeout fields from Config (now always dynamic with 10s upper bound) - DefaultConfig and NewScanner() updated accordingly
Callers can now inject custom output sinks:
type myWriter struct{}
func (w *myWriter) WriteDomainResult(r result.Result) error { ... }
func (w *myWriter) Close() error { return nil }
scanner := sdk.NewScanner(&sdk.Config{
ExtraWriters: []outputter.Output{&myWriter{}},
})
ExtraWriters are appended after the internal primaryWriter so the SDK's
own collect/stream logic is unaffected. Close() is called on each writer
by the runner after the scan finishes.
- New pkg/core/errors package with: ErrPermissionDenied, ErrDeviceNotFound, ErrDeviceNotActive, ErrPcapInit, ErrDomainChanNil - device.go: wrap three fmt.Errorf strings with %w + sentinel - result.go: replace fmt.Errorf string with ErrDomainChanNil - sdk.go: re-export sentinels as package-level vars so callers need only import the sdk package and use errors.Is(err, sdk.ErrPermissionDenied)
- Architecture diagram showing goroutine data flow - Goroutine responsibility table - SDK quick-start with blocking + stream API examples - Config field table (notes that Timeout is not configurable) - Low-level Options field table - Custom output sink guide (outputter.Output interface) - Typed error handling with errors.Is examples - Backpressure description - Build / test commands - Practical notes: one-instance-only, sudo, macOS BPF, WSL2
screen.go: - Remove \r prefix from all output paths (was used to overwrite progress bar in-place, but corrupts piped output) - --od --silent path now emits clean 'domain\n' lines, directly consumable by httpx / grep / awk - Non-silent path pads to terminal width without leading \r - Guard against negative padding width (long domain + narrow terminal) jsonl.go: - Replace per-record file.Sync() with bufio.Writer (64 KiB buffer) to reduce syscall overhead under high throughput - Flush + close in Close(); data is always complete after scanner exits - Extract parseAnswers() helper: handles CNAME/NS/PTR/TXT/AAAA/A prefixes, shared with beautified.go to eliminate duplication - Add AAAA record type detection (was falling through to plain-IP path) - Stable JSON field names: domain / type / records / timestamp beautified.go: - Use shared parseAnswers() instead of inline type-parsing loop
- Add Runner.SuccessCount() uint64 method (reads successCount atomically)
- enum.go + verify.go: call os.Exit(1) after Close() when SuccessCount==0
- Enables correct shell pipeline semantics:
ksubdomain enum -d example.com && httpx ...
(httpx only runs if at least one subdomain was found)
quickstart.md: - Prerequisites, installation, first scan examples - Common flags table, output formats, bandwidth tuning - Quick troubleshooting table api.md: - Full SDK API reference: Config, Result, Scanner methods - Error sentinels with errors.Is examples - Custom output sink interface + example - Complete working example best-practices.md: - Bandwidth selection table by connection type - DNS resolver selection and anti-overload guidance - Wildcard DNS detection and filter-mode guide - Pipe-friendly --silent --od recipe - JSONL + jq analysis examples - Large-scale enumeration tips - SDK cancellation and thread-safety guidance faq.md: - Permissions: sudo vs CAP_NET_RAW - Interface errors: not found, not active, auto-detect - macOS ENOBUFS / BPF buffer - WSL2 interface and libpcap setup - DNS: wildcard, SERVFAIL, incomplete results - httpx garbled input fix - jq JSONL parse errors - Build: libpcap cross-compile - Exit code table
…endCycleWithContext RunEnumeration: goroutine topology diagram showing full data flow from domainChan through statusDB, retry loop, recv pipeline to outputters. sendCycleWithContext: document batching rationale (amortised serialisation cost), 10ms ticker flush, and backpressure protocol with recv.go.
simple/main.go: - Fix Records[0] index-out-of-range (use strings.Join instead) - Add errors.Is checks for ErrPermissionDenied / ErrDeviceNotFound advanced/main.go: - Remove Timeout field (removed from Config) - Use errors.Is for context.DeadlineExceeded and sdk.ErrPermissionDenied - Guard speed calc against zero elapsed .gitignore: add 'simple' and 'advanced' binary names
- Trigger on push to main/dev/feature/** and PRs to main/dev - Build matrix: Linux amd64 (static libpcap), Linux arm64 (CGO=0), macOS amd64, macOS arm64, Windows amd64 (cross-compiled CGO=0) - fail-fast: false so all platform results are visible simultaneously - Separate lint job: go vet + staticcheck (no root/pcap required) - Uses actions/checkout@v4, actions/setup-go@v5 (latest stable)
- pkg/core/conf/config.go: change Version from const to var so ldflags can override it at link time; default value '2.4-dev' distinguishes untagged dev builds from official releases - build.yml: add -ldflags to all three go build invocations, injecting the git tag captured in get_version step - build.sh: rewrite as proper bash script; auto-detect version from 'git describe --tags --always'; support override via $1 argument; add Windows cross-compile step
Revert .github/workflows/build.yml to main version. Remove .github/workflows/ci.yml (added in P3-7). CI matrix and ldflags changes can be applied directly on GitHub or after re-generating a token with 'workflow' scope.
- Add EtherInfos []*device.EtherTable to Options for multi-NIC config - Add GetDeviceConfigs() to support repeated --eth flags - Refactor Runner to manage per-NIC netInterface contexts (pcapHandle, listenPort, templateCache) - Launch one sendCycleForIface goroutine per NIC (A1 pattern): all goroutines compete on shared domainChan for natural load balancing - Launch one recvChanelForIface goroutine per NIC with per-NIC BPF filter - Sink templateCache into netInterface to avoid cross-NIC template reuse - Update --eth/--interface flag to StringSlice for repeated usage - Update SDK Config with Devices []string field - Single-NIC path is fully backward compatible
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
为 ksubdomain 增加多网卡并发发包能力,允许用户通过重复指定
--eth/--interface参数来使用多张网卡同时发包,从而叠加带宽或利用多个出口 IP。实现方案
架构:A1 模式(每卡一个独立 sendCycle goroutine)
每张网卡启动一个独立的
sendCycleForIfacegoroutine,所有 goroutine 共享同一个domainChan,竞争消费域名,天然实现负载均衡,无需额外调度逻辑。改动要点
pkg/core/options/options.goEtherInfos []*device.EtherTable字段,保留EtherInfo向后兼容AllEtherInfos()方法:优先返回EtherInfos,若为空则将EtherInfo包装为单元素切片pkg/core/options/device.goGetDeviceConfigs(ethNames []string, dnsServer []string) []*device.EtherTableGetInterfaceByNamepkg/device/network_improved.goGetInterfaceByName(name string, userDNS []string) (*EtherTable, error)GetDefaultGatewayIP()辅助函数pkg/runner/runner.gonetInterface结构体,封装单张网卡的完整上下文(etherInfo/pcapHandle/listenPort/templateCache)Runner用ifaces []*netInterface替代原有的pcapHandle和listenPort字段New()循环初始化每张网卡的 pcapHandle + freeportRunEnumeration()动态wg.Add(3 + len(ifaces)*2),为每张网卡启动 send/recv goroutine 对Close()循环关闭所有网卡句柄pkg/runner/send.gotemplateCache下沉到netInterface.templateCache,避免跨网卡模板复用(不同网卡 SrcMAC/SrcIP 不同)getOrCreate改为netInterface的方法send()签名改为接受*netInterface,从 iface 上下文取 etherInfo 和 pcapHandlesendCycleWithContext重构为sendCycleForIface(ctx, wg, iface)pkg/runner/recv.gorecvChanel重构为recvChanelForIface(ctx, wg, iface)listenPort,保证收包互不干扰cmd/ksubdomain/verify.go/enum.go--interface/--ethflag 从StringFlag改为StringSliceFlag,支持重复指定ethNames := c.StringSlice("interface"),调用GetDeviceConfigs填充EtherInfossdk/sdk.goConfig新增Devices []string(可选多网卡),buildOptions中适配使用方式
兼容性
EtherInfo字段保留,所有存量代码无需修改go build ./...编译通过,go vet ./...本次改动范围内无新增告警(存量测试文件告警非本次引入)