eth0 with a source IP inside the Kubernetes pod/overlay range (e.g., 10.244.0.0/22) toward the default gateway—an address the node does not own—which looks like source spoofing. This behavior occurs during or just after boot, before the CNI (cni0/flannel) network is fully up.VMware Cloud Foundation (VCF) 9.0.x, 9.1.x
Log Management (formerly Aria Operations for Logs)
External Aria Orchestrator 8.18.1 / 9.0.2
Outbound anti-spoofing is not enforced by the firewall itself; it is a side effect of Flannel's --ip-masq POSTROUTING rules. Flannel runs as a Kubernetes pod that comes up late in the boot process. From the moment eth0 is up until flanneld installs those rules, there is a window in which a cluster-sourced packet can leave eth0 without being SNAT'd. The base ruleset (/opt/scripts/iptables/start) has :OUTPUT ACCEPT and only drops non-cluster sources (! -s clusterCIDR), so nothing masquerades or blocks cluster-sourced egress during that window. Because /opt/scripts/iptables/start rebuilds the nat table with iptables-restore without --noflush, re-applying it also transiently drops Flannel's rules and reopens the window until Flannel's next resync.
These instructions walk you through updating the base firewall ruleset, ensuring all cluster-sourced traffic leaving eth0 is SNAT'd to the node IP properly from the moment the appliance starts up.
The patch adds the following to /opt/scripts/iptables/start:
clusterCIDR=$(vracli network k8s-subnets --cluster-cidr) # before the heredocAdd a new chain declaration in the *nat table:
:PRELUDE_EGRESS_MASQ - [0:0]After the existing loopback MASQUERADE rule, add:
-A POSTROUTING -s ${clusterCIDR} -o eth0 -j PRELUDE_EGRESS_MASQ
-A PRELUDE_EGRESS_MASQ -d ${clusterCIDR} -j RETURN
-A PRELUDE_EGRESS_MASQ -d 224.0.0.0/4 -j RETURN
-A PRELUDE_EGRESS_MASQ -j MASQUERADE --random-fullyThe RETURN and MASQUERADE rules live in a dedicated chain (PRELUDE_EGRESS_MASQ) rather than inline in POSTROUTING. This ensures kube-proxy's and Flannel's own independent reconciliation of POSTROUTING can never reorder them (see "Additional notes" for why that matters). A permanent fix is tentatively planned to be included in a future release; until then, apply the workaround below.
Complete the following steps:
Take simultaneous VM snapshots (without memory, with the quiesced option selected) of all nodes.
Run the following command on a single node to apply and persist the fix on all nodes:
vracli cluster exec -- bash -c 'base64 -d <<< "IyEvYmluL2Jhc2gKIwojIFZDT1BTLTkzMjk4IC0gRWdyZXNzIGFudGktc3Bvb2YgbWFzcXVlcmFkZSBob3RmaXggZm9yIHRoZSBWQSBmaXJld2FsbC4KIwojIEJhY2tncm91bmQsIGRlc2lnbiByYXRpb25hbGUsIGFuZCBmdWxsIGluc3RhbGwvdGVzdC9yb2xsYmFjayBzdGVwczoKIyBzZWUgdGhlIFJFQURNRSBuZXh0IHRvIHRoaXMgc2NyaXB0LgojCiMgVGhpcyBpcyBwZXItbm9kZSBsb2dpYy4gUnVuIGl0IHZpYSAndnJhY2xpIGNsdXN0ZXIgZXhlYyAtLSBiYXNoIDx0aGlzIGZpbGU+JwojIChvciB0aGUgYmFzZTY0IG9uZS1saW5lciBpbiB0aGUgUkVBRE1FKSBzbyBpdCBhcHBsaWVzIG9uIGV2ZXJ5IG5vZGUgLQojIHJ1bm5pbmcgaXQgZGlyZWN0bHkgb25seSBwYXRjaGVzIHRoZSBsb2NhbCBub2RlLgojCiMgVEFLRSBTSU1VTFRBTkVPVVMsIFFVSUVTQ0VEIFZNIFNOQVBTSE9UUyAoTk8gTUVNT1JZKSBPRiBBTEwgTk9ERVMgRklSU1QuCiMKc2V0IC11byBwaXBlZmFpbAoKUFI9IlZDT1BTLTkzMjk4IgpDSEFJTj0iUFJFTFVERV9FR1JFU1NfTUFTUSIKVEFSR0VUPSIvb3B0L3NjcmlwdHMvaXB0YWJsZXMvc3RhcnQiClRTPSIkKGRhdGUgJyslWS0lbS0lZC0lSC0lTS0lUycpIgpCQUNLVVBfRElSPSIvZGF0YS9wYXRjaC1iYWNrdXAvJHtQUn0tJHtUU30iCmxvZ0Rpcj0iL3Zhci9sb2cvdm13YXJlL3ByZWx1ZGUiCm1rZGlyIC1wICIkbG9nRGlyIgpleGVjIDI+ID4odGVlIC1hICIkbG9nRGlyL3BhdGNoLSR7UFJ9LSR7VFN9LmxvZyIpCmV4ZWMgPiYyCgpIPSIkKGhvc3RuYW1lKSIKbG9nKCl7IGVjaG8gIlskMV1bJChkYXRlICcrJVktJW0tJWQtJUgtJU0tJVMnKV1bJHtIfV0gJDIiOyB9CgppZiBbICIkKGlkIC11KSIgLW5lIDAgXTsgdGhlbiBsb2cgRVJST1IgIk5vdCBydW5uaW5nIGFzIHJvb3QuIjsgZXhpdCAxOyBmaQppZiBbICEgLWYgIiRUQVJHRVQiIF07IHRoZW4gbG9nIEVSUk9SICIkVEFSR0VUIG5vdCBmb3VuZDsgdW5zdXBwb3J0ZWQgbm9kZSBsYXlvdXQuIjsgZXhpdCAxOyBmaQoKIyBJZGVtcG90ZW5jeTogc2tpcCBpZiBhbHJlYWR5IHBhdGNoZWQgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppZiBncmVwIC1xICIkQ0hBSU4iICIkVEFSR0VUIjsgdGhlbgogIGxvZyBJTkZPICJBbHJlYWR5IHBhdGNoZWQgKCR7Q0hBSU59IGNoYWluIHByZXNlbnQgaW4gJHtUQVJHRVR9KTsgbm90aGluZyB0byBkby4iCiAgZXhpdCAwCmZpCgojIEFuY2hvciBzYW5pdHkgKGJvdGggc3RhYmxlIGFjcm9zcyBzaGlwcGluZyBhbmQgVlJBRS02NjA0NC1wYXRjaGVkIHZhcmlhbnRzKQppZiAhIGdyZXAgLXEgJ2lwdGFibGVzLXJlc3RvcmUgLS1jb3VudGVycyA8PCBfX0VPRl9fJyAiJFRBUkdFVCI7IHRoZW4KICBsb2cgRVJST1IgIkFuY2hvciAnaXB0YWJsZXMtcmVzdG9yZSAtLWNvdW50ZXJzIDw8IF9fRU9GX18nIG5vdCBmb3VuZCBpbiAke1RBUkdFVH07IGFib3J0aW5nICh1bmV4cGVjdGVkIGxheW91dCkuIgogIGV4aXQgMQpmaQppZiAhIGdyZXAgLXFFICdeOlBSRUxVREVfUFJPWFkgLSBcWzA6MFxdJCcgIiRUQVJHRVQiOyB0aGVuCiAgbG9nIEVSUk9SICJBbmNob3IgJzpQUkVMVURFX1BST1hZIC0gWzA6MF0nIG5vdCBmb3VuZCBpbiAke1RBUkdFVH07IGFib3J0aW5nICh1bmV4cGVjdGVkIGxheW91dCkuIgogIGV4aXQgMQpmaQppZiAhIGdyZXAgLXEgJ14tQSBQT1NUUk9VVElORyAtbyBldGgwIC0tc3JjIDEyN1wuMFwuMFwuMC84IC1qIE1BU1FVRVJBREUnICIkVEFSR0VUIjsgdGhlbgogIGxvZyBFUlJPUiAiQW5jaG9yIGxvb3BiYWNrIE1BU1FVRVJBREUgcnVsZSBub3QgZm91bmQgaW4gJHtUQVJHRVR9OyBhYm9ydGluZyAodW5leHBlY3RlZCBsYXlvdXQpLiIKICBleGl0IDEKZmkKCiMgQmFja3VwIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpta2RpciAtcCAiJHtCQUNLVVBfRElSfS9vcHQvc2NyaXB0cy9pcHRhYmxlcyIKY3AgLWEgIiRUQVJHRVQiICIke0JBQ0tVUF9ESVJ9L29wdC9zY3JpcHRzL2lwdGFibGVzLyIgfHwgeyBsb2cgRVJST1IgIkJhY2t1cCBmYWlsZWQ7IGFib3J0aW5nLiI7IGV4aXQgMTsgfQpsb2cgSU5GTyAiQmFja2VkIHVwICR7VEFSR0VUfSAtPiAke0JBQ0tVUF9ESVJ9L29wdC9zY3JpcHRzL2lwdGFibGVzL3N0YXJ0IgoKIyBQYXRjaCBpbnRvIGEgdGVtcCBmaWxlIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tClRNUD0iJChta3RlbXApIgphd2sgJwogIC9pcHRhYmxlcy1yZXN0b3JlIC0tY291bnRlcnMgPDwgX19FT0ZfXy8gJiYgIXNlZW5fdmFyIHsKICAgIHByaW50ICJjbHVzdGVyQ0lEUj0kKHZyYWNsaSBuZXR3b3JrIGs4cy1zdWJuZXRzIC0tY2x1c3Rlci1jaWRyKSIKICAgIHByaW50ICIiCiAgICBzZWVuX3Zhcj0xCiAgfQogIHsgcHJpbnQgfQogIC9eOlBSRUxVREVfUFJPWFkgLSBcWzA6MFxdJC8gJiYgIXNlZW5fY2hhaW4gewogICAgcHJpbnQgIjpQUkVMVURFX0VHUkVTU19NQVNRIC0gWzA6MF0iCiAgICBzZWVuX2NoYWluPTEKICB9CiAgL14tQSBQT1NUUk9VVElORyAtbyBldGgwIC0tc3JjIDEyN1wuMFwuMFwuMFwvOCAtaiBNQVNRVUVSQURFLyAmJiAhc2Vlbl9ydWxlIHsKICAgIHByaW50ICIiCiAgICBwcmludCAiLUEgUE9TVFJPVVRJTkcgLXMgJHtjbHVzdGVyQ0lEUn0gLW8gZXRoMCAtaiBQUkVMVURFX0VHUkVTU19NQVNRIgogICAgcHJpbnQgIi1BIFBSRUxVREVfRUdSRVNTX01BU1EgLWQgJHtjbHVzdGVyQ0lEUn0gLWogUkVUVVJOIgogICAgcHJpbnQgIi1BIFBSRUxVREVfRUdSRVNTX01BU1EgLWQgMjI0LjAuMC4wLzQgLWogUkVUVVJOIgogICAgcHJpbnQgIi1BIFBSRUxVREVfRUdSRVNTX01BU1EgLWogTUFTUVVFUkFERSAtLXJhbmRvbS1mdWxseSIKICAgIHNlZW5fcnVsZT0xCiAgfQonICIkVEFSR0VUIiA+ICIkVE1QIgoKIyBDb25maXJtIGFsbCB0aHJlZSBpbnNlcnRpb25zIGxhbmRlZCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppZiAhIGdyZXAgLXFGICdjbHVzdGVyQ0lEUj0kKHZyYWNsaSBuZXR3b3JrIGs4cy1zdWJuZXRzIC0tY2x1c3Rlci1jaWRyKScgIiRUTVAiIFwKICAgfHwgISBncmVwIC1xRiAnOlBSRUxVREVfRUdSRVNTX01BU1EgLSBbMDowXScgIiRUTVAiIFwKICAgfHwgISBncmVwIC1xRiAnTUFTUVVFUkFERSAtLXJhbmRvbS1mdWxseScgIiRUTVAiOyB0aGVuCiAgbG9nIEVSUk9SICJQYXRjaCBkaWQgbm90IGFwcGx5IGNsZWFubHkgKGV4cGVjdGVkIGluc2VydGlvbnMgbWlzc2luZykuIExlYXZpbmcgJHtUQVJHRVR9IHVudG91Y2hlZC4iCiAgcm0gLWYgIiRUTVAiOyBleGl0IDEKZmkKCiMgVmFsaWRhdGUgc2hlbGwgc3ludGF4IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppZiAhIHNoIC1uICIkVE1QIjsgdGhlbgogIGxvZyBFUlJPUiAiUGF0Y2hlZCBzY3JpcHQgZmFpbGVkIHNoZWxsIHN5bnRheCBjaGVjay4gTGVhdmluZyAke1RBUkdFVH0gdW50b3VjaGVkLiIKICBybSAtZiAiJFRNUCI7IGV4aXQgMQpmaQoKIyBWYWxpZGF0ZSB0aGUgbmV3IG5hdCBydWxlcyBhZ2FpbnN0IHRoZSBydW5uaW5nIGtlcm5lbCAoZHJ5LXJ1biwgbm8gY2hhbmdlcykKQ0M9IiQodnJhY2xpIG5ldHdvcmsgazhzLXN1Ym5ldHMgLS1jbHVzdGVyLWNpZHIgfHwgdHJ1ZSkiCmlmIFsgLXogIiRDQyIgXTsgdGhlbgogIGxvZyBFUlJPUiAiQ291bGQgbm90IGRldGVybWluZSBjbHVzdGVyIENJRFIgKHZyYWNsaSByZXR1cm5lZCBlbXB0eSkuIExlYXZpbmcgJHtUQVJHRVR9IHVudG91Y2hlZC4iCiAgcm0gLWYgIiRUTVAiOyBleGl0IDEKZmkKaWYgISBwcmludGYgJypuYXRcbjolcyAtIFswOjBdXG4tQSBQT1NUUk9VVElORyAtcyAlcyAtbyBldGgwIC1qICVzXG4tQSAlcyAtZCAlcyAtaiBSRVRVUk5cbi1BICVzIC1kIDIyNC4wLjAuMC80IC1qIFJFVFVSTlxuLUEgJXMgLWogTUFTUVVFUkFERSAtLXJhbmRvbS1mdWxseVxuQ09NTUlUXG4nIFwKICAgICAgIiRDSEFJTiIgIiRDQyIgIiRDSEFJTiIgIiRDSEFJTiIgIiRDQyIgIiRDSEFJTiIgIiRDSEFJTiIgfCBpcHRhYmxlcy1yZXN0b3JlIC0tdGVzdCAtLW5vZmx1c2g7IHRoZW4KICBsb2cgRVJST1IgIk5ldyBuYXQgcnVsZXMgZmFpbGVkICdpcHRhYmxlcy1yZXN0b3JlIC0tdGVzdCcgZm9yIGNsdXN0ZXIgQ0lEUiAke0NDfS4gTGVhdmluZyAke1RBUkdFVH0gdW50b3VjaGVkLiIKICBybSAtZiAiJFRNUCI7IGV4aXQgMQpmaQpsb2cgSU5GTyAiRHJ5LXJ1biB2YWxpZGF0aW9uIE9LIChzaGVsbCBzeW50YXggKyB0ZXN0LWxvYWQgZm9yIGNsdXN0ZXIgQ0lEUiAke0NDfSkuIgpsb2cgSU5GTyAiTm90ZTogYSBkcnktcnVuIHRlc3Qgb25seSBjb25maXJtcyB0aGUgcnVsZXMgYXJlIHdlbGwtZm9ybWVkIGFuZCBsb2FkYWJsZSAtLSBpdCBkb2VzIgpsb2cgSU5GTyAiTk9UIGNvbmZpcm0gcnVudGltZSBvcmRlcmluZy4gVGhlIGxpdmUgcG9zdC1yZWxvYWQgY2hlY2sgYmVsb3cgaXMgdGhlIHJlYWwgZ2F0ZS4iCgojIEluc3RhbGwgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KaW5zdGFsbCAtbSAwNzU1ICIkVE1QIiAiJFRBUkdFVCIgfHwgeyBsb2cgRVJST1IgIkluc3RhbGwgdG8gJHtUQVJHRVR9IGZhaWxlZDsgcmVzdG9yZSBmcm9tICR7QkFDS1VQX0RJUn0uIjsgcm0gLWYgIiRUTVAiOyBleGl0IDE7IH0Kcm0gLWYgIiRUTVAiCmxvZyBJTkZPICJQYXRjaGVkICR7VEFSR0VUfS4iCgojIEFwcGx5IGxpdmUgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0Kcm9sbGJhY2soKXsKICBsb2cgRVJST1IgIlJvbGxpbmcgdGhpcyBub2RlIGJhY2sgdG8gaXRzIHByZS1wYXRjaCBiYWNrdXAuLi4iCiAgY3AgLWEgIiR7QkFDS1VQX0RJUn0vb3B0L3NjcmlwdHMvaXB0YWJsZXMvc3RhcnQiICIkVEFSR0VUIgogIHN5c3RlbWN0bCByZWxvYWQgaXB0YWJsZXMKICBsb2cgRVJST1IgIlJvbGxlZCBiYWNrIGFuZCByZWxvYWRlZC4gJHtUQVJHRVR9IGlzIGJhY2sgdG8gaXRzIHByZS1wYXRjaCBzdGF0ZSBvbiAke0h9LiIKfQoKaWYgISBzeXN0ZW1jdGwgcmVsb2FkIGlwdGFibGVzOyB0aGVuCiAgbG9nIEVSUk9SICInc3lzdGVtY3RsIHJlbG9hZCBpcHRhYmxlcycgRkFJTEVELiBpcHRhYmxlcy1yZXN0b3JlIGlzIGF0b21pYywgc28gdGhlIExJVkUgcnVsZXNldCBpcyB1bmNoYW5nZWQsIgogIGxvZyBFUlJPUiAiYnV0ICR7VEFSR0VUfSBpcyBwYXRjaGVkIG9uIGRpc2suIFJvbGxpbmcgdGhlIGZpbGUgYmFjayB0byBhdm9pZCBhIG1pc21hdGNoIGJldHdlZW4gZGlzayBhbmQga2VybmVsLiIKICBjcCAtYSAiJHtCQUNLVVBfRElSfS9vcHQvc2NyaXB0cy9pcHRhYmxlcy9zdGFydCIgIiRUQVJHRVQiCiAgZXhpdCAxCmZpCmxvZyBJTkZPICJpcHRhYmxlcyByZWxvYWRlZC4gVmVyaWZ5aW5nIHRoZSBMSVZFIHJ1bGVzZXQgYWN0dWFsbHkgbWF0Y2hlcyB3aGF0IHdhcyBpbnRlbmRlZC4uLiIKCiMgTGl2ZSB2ZXJpZmljYXRpb24gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiMgQ29tcGFyZSB0aGUgcnVubmluZyBrZXJuZWwncyBQUkVMVURFX0VHUkVTU19NQVNRIGNoYWluLCBydWxlLWZvci1ydWxlIGFuZCBpbgojIG9yZGVyLCBhZ2FpbnN0IGV4YWN0bHkgd2hhdCB3ZSBleHBlY3QuIEEgZHJ5LXJ1biB0ZXN0IGNhbm5vdCBjYXRjaCBydW50aW1lCiMgcmVvcmRlcmluZyBieSBvdGhlciByZWNvbmNpbGVyczsgb25seSBpbnNwZWN0aW5nIHRoZSBsaXZlIHRhYmxlIGFmdGVyIGEgcmVhbAojIHJlbG9hZCBjYW4uIFNlZSB0aGUgUkVBRE1FIGZvciB3aHkgdGhpcyBjaGVjayBleGlzdHMuCkVYUEVDVEVEX0NIQUlOPSIkKHByaW50ZiAtLSAnLU4gJXNcbi1BICVzIC1kICVzIC1qIFJFVFVSTlxuLUEgJXMgLWQgMjI0LjAuMC4wLzQgLWogUkVUVVJOXG4tQSAlcyAtaiBNQVNRVUVSQURFIC0tcmFuZG9tLWZ1bGx5JyBcCiAgIiRDSEFJTiIgIiRDSEFJTiIgIiRDQyIgIiRDSEFJTiIgIiRDSEFJTiIpIgpBQ1RVQUxfQ0hBSU49IiQoaXB0YWJsZXMgLXQgbmF0IC1TICIkQ0hBSU4iIDI+L2Rldi9udWxsKSIKCmlmIFsgIiRFWFBFQ1RFRF9DSEFJTiIgIT0gIiRBQ1RVQUxfQ0hBSU4iIF07IHRoZW4KICBsb2cgRVJST1IgIkxpdmUgY2hhaW4gJHtDSEFJTn0gZG9lcyBub3QgbWF0Y2ggd2hhdCB3YXMgZXhwZWN0ZWQgYWZ0ZXIgcmVsb2FkLiIKICBsb2cgRVJST1IgIkV4cGVjdGVkOiIKICBlY2hvICIkRVhQRUNURURfQ0hBSU4iIHwgc2VkICdzL14vICAgIC8nID4mMgogIGxvZyBFUlJPUiAiQWN0dWFsOiIKICBlY2hvICIke0FDVFVBTF9DSEFJTjotPGNoYWluIG5vdCBmb3VuZD59IiB8IHNlZCAncy9eLyAgICAvJyA+JjIKICByb2xsYmFjawogIGV4aXQgMQpmaQoKaWYgISBpcHRhYmxlcyAtdCBuYXQgLUMgUE9TVFJPVVRJTkcgLXMgIiRDQyIgLW8gZXRoMCAtaiAiJENIQUlOIiAyPi9kZXYvbnVsbDsgdGhlbgogIGxvZyBFUlJPUiAiTGl2ZSBQT1NUUk9VVElORyBqdW1wICctcyAke0NDfSAtbyBldGgwIC1qICR7Q0hBSU59JyBpcyBtaXNzaW5nIGFmdGVyIHJlbG9hZC4iCiAgcm9sbGJhY2sKICBleGl0IDEKZmkKCmxvZyBJTkZPICJMaXZlIHZlcmlmaWNhdGlvbiBPSzogJHtDSEFJTn0gZXhpc3RzIHdpdGggZXhhY3RseSB0aGUgZXhwZWN0ZWQgcnVsZXMsIGluIG9yZGVyLCBhbmQiCmxvZyBJTkZPICJQT1NUUk9VVElORyBqdW1wcyB0byBpdCBjb3JyZWN0bHkuIgpsb2cgSU5GTyAiJHtQUn0gY29tcGxldGUgb24gJHtIfS4iCg==" | bash -'Running the script via vracli cluster exec applies it on every node. Per node, it backs up /opt/scripts/iptables/start under /data/patch-backup/VCOPS-93298-<timestamp>/, skips the process (idempotent) if the fix is already applied, patches the file, and validates it (using shell syntax and iptables-restore --test) BEFORE touching the live file. It then reloads iptables, verifies the live kernel ruleset matches exactly what was intended, and automatically rolls that node back to its pre-patch backup if it does not.
Log location: /var/log/vmware/prelude/patch-VCOPS-93298-<timestamp>.log
Before running the script above, you can review the script using the command below. This will confirm that the base64 encoded script is being copied and pasted correctly.
base64 -d <<< "IyEvYmluL2Jhc2gKIwojIFZDT1BTLTkzMjk4IC0gRWdyZXNzIGFudGktc3Bvb2YgbWFzcXVlcmFkZSBob3RmaXggZm9yIHRoZSBWQSBmaXJld2FsbC4KIwojIEJhY2tncm91bmQsIGRlc2lnbiByYXRpb25hbGUsIGFuZCBmdWxsIGluc3RhbGwvdGVzdC9yb2xsYmFjayBzdGVwczoKIyBzZWUgdGhlIFJFQURNRSBuZXh0IHRvIHRoaXMgc2NyaXB0LgojCiMgVGhpcyBpcyBwZXItbm9kZSBsb2dpYy4gUnVuIGl0IHZpYSAndnJhY2xpIGNsdXN0ZXIgZXhlYyAtLSBiYXNoIDx0aGlzIGZpbGU+JwojIChvciB0aGUgYmFzZTY0IG9uZS1saW5lciBpbiB0aGUgUkVBRE1FKSBzbyBpdCBhcHBsaWVzIG9uIGV2ZXJ5IG5vZGUgLQojIHJ1bm5pbmcgaXQgZGlyZWN0bHkgb25seSBwYXRjaGVzIHRoZSBsb2NhbCBub2RlLgojCiMgVEFLRSBTSU1VTFRBTkVPVVMsIFFVSUVTQ0VEIFZNIFNOQVBTSE9UUyAoTk8gTUVNT1JZKSBPRiBBTEwgTk9ERVMgRklSU1QuCiMKc2V0IC11byBwaXBlZmFpbAoKUFI9IlZDT1BTLTkzMjk4IgpDSEFJTj0iUFJFTFVERV9FR1JFU1NfTUFTUSIKVEFSR0VUPSIvb3B0L3NjcmlwdHMvaXB0YWJsZXMvc3RhcnQiClRTPSIkKGRhdGUgJyslWS0lbS0lZC0lSC0lTS0lUycpIgpCQUNLVVBfRElSPSIvZGF0YS9wYXRjaC1iYWNrdXAvJHtQUn0tJHtUU30iCmxvZ0Rpcj0iL3Zhci9sb2cvdm13YXJlL3ByZWx1ZGUiCm1rZGlyIC1wICIkbG9nRGlyIgpleGVjIDI+ID4odGVlIC1hICIkbG9nRGlyL3BhdGNoLSR7UFJ9LSR7VFN9LmxvZyIpCmV4ZWMgPiYyCgpIPSIkKGhvc3RuYW1lKSIKbG9nKCl7IGVjaG8gIlskMV1bJChkYXRlICcrJVktJW0tJWQtJUgtJU0tJVMnKV1bJHtIfV0gJDIiOyB9CgppZiBbICIkKGlkIC11KSIgLW5lIDAgXTsgdGhlbiBsb2cgRVJST1IgIk5vdCBydW5uaW5nIGFzIHJvb3QuIjsgZXhpdCAxOyBmaQppZiBbICEgLWYgIiRUQVJHRVQiIF07IHRoZW4gbG9nIEVSUk9SICIkVEFSR0VUIG5vdCBmb3VuZDsgdW5zdXBwb3J0ZWQgbm9kZSBsYXlvdXQuIjsgZXhpdCAxOyBmaQoKIyBJZGVtcG90ZW5jeTogc2tpcCBpZiBhbHJlYWR5IHBhdGNoZWQgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppZiBncmVwIC1xICIkQ0hBSU4iICIkVEFSR0VUIjsgdGhlbgogIGxvZyBJTkZPICJBbHJlYWR5IHBhdGNoZWQgKCR7Q0hBSU59IGNoYWluIHByZXNlbnQgaW4gJHtUQVJHRVR9KTsgbm90aGluZyB0byBkby4iCiAgZXhpdCAwCmZpCgojIEFuY2hvciBzYW5pdHkgKGJvdGggc3RhYmxlIGFjcm9zcyBzaGlwcGluZyBhbmQgVlJBRS02NjA0NC1wYXRjaGVkIHZhcmlhbnRzKQppZiAhIGdyZXAgLXEgJ2lwdGFibGVzLXJlc3RvcmUgLS1jb3VudGVycyA8PCBfX0VPRl9fJyAiJFRBUkdFVCI7IHRoZW4KICBsb2cgRVJST1IgIkFuY2hvciAnaXB0YWJsZXMtcmVzdG9yZSAtLWNvdW50ZXJzIDw8IF9fRU9GX18nIG5vdCBmb3VuZCBpbiAke1RBUkdFVH07IGFib3J0aW5nICh1bmV4cGVjdGVkIGxheW91dCkuIgogIGV4aXQgMQpmaQppZiAhIGdyZXAgLXFFICdeOlBSRUxVREVfUFJPWFkgLSBcWzA6MFxdJCcgIiRUQVJHRVQiOyB0aGVuCiAgbG9nIEVSUk9SICJBbmNob3IgJzpQUkVMVURFX1BST1hZIC0gWzA6MF0nIG5vdCBmb3VuZCBpbiAke1RBUkdFVH07IGFib3J0aW5nICh1bmV4cGVjdGVkIGxheW91dCkuIgogIGV4aXQgMQpmaQppZiAhIGdyZXAgLXEgJ14tQSBQT1NUUk9VVElORyAtbyBldGgwIC0tc3JjIDEyN1wuMFwuMFwuMC84IC1qIE1BU1FVRVJBREUnICIkVEFSR0VUIjsgdGhlbgogIGxvZyBFUlJPUiAiQW5jaG9yIGxvb3BiYWNrIE1BU1FVRVJBREUgcnVsZSBub3QgZm91bmQgaW4gJHtUQVJHRVR9OyBhYm9ydGluZyAodW5leHBlY3RlZCBsYXlvdXQpLiIKICBleGl0IDEKZmkKCiMgQmFja3VwIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpta2RpciAtcCAiJHtCQUNLVVBfRElSfS9vcHQvc2NyaXB0cy9pcHRhYmxlcyIKY3AgLWEgIiRUQVJHRVQiICIke0JBQ0tVUF9ESVJ9L29wdC9zY3JpcHRzL2lwdGFibGVzLyIgfHwgeyBsb2cgRVJST1IgIkJhY2t1cCBmYWlsZWQ7IGFib3J0aW5nLiI7IGV4aXQgMTsgfQpsb2cgSU5GTyAiQmFja2VkIHVwICR7VEFSR0VUfSAtPiAke0JBQ0tVUF9ESVJ9L29wdC9zY3JpcHRzL2lwdGFibGVzL3N0YXJ0IgoKIyBQYXRjaCBpbnRvIGEgdGVtcCBmaWxlIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tClRNUD0iJChta3RlbXApIgphd2sgJwogIC9pcHRhYmxlcy1yZXN0b3JlIC0tY291bnRlcnMgPDwgX19FT0ZfXy8gJiYgIXNlZW5fdmFyIHsKICAgIHByaW50ICJjbHVzdGVyQ0lEUj0kKHZyYWNsaSBuZXR3b3JrIGs4cy1zdWJuZXRzIC0tY2x1c3Rlci1jaWRyKSIKICAgIHByaW50ICIiCiAgICBzZWVuX3Zhcj0xCiAgfQogIHsgcHJpbnQgfQogIC9eOlBSRUxVREVfUFJPWFkgLSBcWzA6MFxdJC8gJiYgIXNlZW5fY2hhaW4gewogICAgcHJpbnQgIjpQUkVMVURFX0VHUkVTU19NQVNRIC0gWzA6MF0iCiAgICBzZWVuX2NoYWluPTEKICB9CiAgL14tQSBQT1NUUk9VVElORyAtbyBldGgwIC0tc3JjIDEyN1wuMFwuMFwuMFwvOCAtaiBNQVNRVUVSQURFLyAmJiAhc2Vlbl9ydWxlIHsKICAgIHByaW50ICIiCiAgICBwcmludCAiLUEgUE9TVFJPVVRJTkcgLXMgJHtjbHVzdGVyQ0lEUn0gLW8gZXRoMCAtaiBQUkVMVURFX0VHUkVTU19NQVNRIgogICAgcHJpbnQgIi1BIFBSRUxVREVfRUdSRVNTX01BU1EgLWQgJHtjbHVzdGVyQ0lEUn0gLWogUkVUVVJOIgogICAgcHJpbnQgIi1BIFBSRUxVREVfRUdSRVNTX01BU1EgLWQgMjI0LjAuMC4wLzQgLWogUkVUVVJOIgogICAgcHJpbnQgIi1BIFBSRUxVREVfRUdSRVNTX01BU1EgLWogTUFTUVVFUkFERSAtLXJhbmRvbS1mdWxseSIKICAgIHNlZW5fcnVsZT0xCiAgfQonICIkVEFSR0VUIiA+ICIkVE1QIgoKIyBDb25maXJtIGFsbCB0aHJlZSBpbnNlcnRpb25zIGxhbmRlZCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppZiAhIGdyZXAgLXFGICdjbHVzdGVyQ0lEUj0kKHZyYWNsaSBuZXR3b3JrIGs4cy1zdWJuZXRzIC0tY2x1c3Rlci1jaWRyKScgIiRUTVAiIFwKICAgfHwgISBncmVwIC1xRiAnOlBSRUxVREVfRUdSRVNTX01BU1EgLSBbMDowXScgIiRUTVAiIFwKICAgfHwgISBncmVwIC1xRiAnTUFTUVVFUkFERSAtLXJhbmRvbS1mdWxseScgIiRUTVAiOyB0aGVuCiAgbG9nIEVSUk9SICJQYXRjaCBkaWQgbm90IGFwcGx5IGNsZWFubHkgKGV4cGVjdGVkIGluc2VydGlvbnMgbWlzc2luZykuIExlYXZpbmcgJHtUQVJHRVR9IHVudG91Y2hlZC4iCiAgcm0gLWYgIiRUTVAiOyBleGl0IDEKZmkKCiMgVmFsaWRhdGUgc2hlbGwgc3ludGF4IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppZiAhIHNoIC1uICIkVE1QIjsgdGhlbgogIGxvZyBFUlJPUiAiUGF0Y2hlZCBzY3JpcHQgZmFpbGVkIHNoZWxsIHN5bnRheCBjaGVjay4gTGVhdmluZyAke1RBUkdFVH0gdW50b3VjaGVkLiIKICBybSAtZiAiJFRNUCI7IGV4aXQgMQpmaQoKIyBWYWxpZGF0ZSB0aGUgbmV3IG5hdCBydWxlcyBhZ2FpbnN0IHRoZSBydW5uaW5nIGtlcm5lbCAoZHJ5LXJ1biwgbm8gY2hhbmdlcykKQ0M9IiQodnJhY2xpIG5ldHdvcmsgazhzLXN1Ym5ldHMgLS1jbHVzdGVyLWNpZHIgfHwgdHJ1ZSkiCmlmIFsgLXogIiRDQyIgXTsgdGhlbgogIGxvZyBFUlJPUiAiQ291bGQgbm90IGRldGVybWluZSBjbHVzdGVyIENJRFIgKHZyYWNsaSByZXR1cm5lZCBlbXB0eSkuIExlYXZpbmcgJHtUQVJHRVR9IHVudG91Y2hlZC4iCiAgcm0gLWYgIiRUTVAiOyBleGl0IDEKZmkKaWYgISBwcmludGYgJypuYXRcbjolcyAtIFswOjBdXG4tQSBQT1NUUk9VVElORyAtcyAlcyAtbyBldGgwIC1qICVzXG4tQSAlcyAtZCAlcyAtaiBSRVRVUk5cbi1BICVzIC1kIDIyNC4wLjAuMC80IC1qIFJFVFVSTlxuLUEgJXMgLWogTUFTUVVFUkFERSAtLXJhbmRvbS1mdWxseVxuQ09NTUlUXG4nIFwKICAgICAgIiRDSEFJTiIgIiRDQyIgIiRDSEFJTiIgIiRDSEFJTiIgIiRDQyIgIiRDSEFJTiIgIiRDSEFJTiIgfCBpcHRhYmxlcy1yZXN0b3JlIC0tdGVzdCAtLW5vZmx1c2g7IHRoZW4KICBsb2cgRVJST1IgIk5ldyBuYXQgcnVsZXMgZmFpbGVkICdpcHRhYmxlcy1yZXN0b3JlIC0tdGVzdCcgZm9yIGNsdXN0ZXIgQ0lEUiAke0NDfS4gTGVhdmluZyAke1RBUkdFVH0gdW50b3VjaGVkLiIKICBybSAtZiAiJFRNUCI7IGV4aXQgMQpmaQpsb2cgSU5GTyAiRHJ5LXJ1biB2YWxpZGF0aW9uIE9LIChzaGVsbCBzeW50YXggKyB0ZXN0LWxvYWQgZm9yIGNsdXN0ZXIgQ0lEUiAke0NDfSkuIgpsb2cgSU5GTyAiTm90ZTogYSBkcnktcnVuIHRlc3Qgb25seSBjb25maXJtcyB0aGUgcnVsZXMgYXJlIHdlbGwtZm9ybWVkIGFuZCBsb2FkYWJsZSAtLSBpdCBkb2VzIgpsb2cgSU5GTyAiTk9UIGNvbmZpcm0gcnVudGltZSBvcmRlcmluZy4gVGhlIGxpdmUgcG9zdC1yZWxvYWQgY2hlY2sgYmVsb3cgaXMgdGhlIHJlYWwgZ2F0ZS4iCgojIEluc3RhbGwgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KaW5zdGFsbCAtbSAwNzU1ICIkVE1QIiAiJFRBUkdFVCIgfHwgeyBsb2cgRVJST1IgIkluc3RhbGwgdG8gJHtUQVJHRVR9IGZhaWxlZDsgcmVzdG9yZSBmcm9tICR7QkFDS1VQX0RJUn0uIjsgcm0gLWYgIiRUTVAiOyBleGl0IDE7IH0Kcm0gLWYgIiRUTVAiCmxvZyBJTkZPICJQYXRjaGVkICR7VEFSR0VUfS4iCgojIEFwcGx5IGxpdmUgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0Kcm9sbGJhY2soKXsKICBsb2cgRVJST1IgIlJvbGxpbmcgdGhpcyBub2RlIGJhY2sgdG8gaXRzIHByZS1wYXRjaCBiYWNrdXAuLi4iCiAgY3AgLWEgIiR7QkFDS1VQX0RJUn0vb3B0L3NjcmlwdHMvaXB0YWJsZXMvc3RhcnQiICIkVEFSR0VUIgogIHN5c3RlbWN0bCByZWxvYWQgaXB0YWJsZXMKICBsb2cgRVJST1IgIlJvbGxlZCBiYWNrIGFuZCByZWxvYWRlZC4gJHtUQVJHRVR9IGlzIGJhY2sgdG8gaXRzIHByZS1wYXRjaCBzdGF0ZSBvbiAke0h9LiIKfQoKaWYgISBzeXN0ZW1jdGwgcmVsb2FkIGlwdGFibGVzOyB0aGVuCiAgbG9nIEVSUk9SICInc3lzdGVtY3RsIHJlbG9hZCBpcHRhYmxlcycgRkFJTEVELiBpcHRhYmxlcy1yZXN0b3JlIGlzIGF0b21pYywgc28gdGhlIExJVkUgcnVsZXNldCBpcyB1bmNoYW5nZWQsIgogIGxvZyBFUlJPUiAiYnV0ICR7VEFSR0VUfSBpcyBwYXRjaGVkIG9uIGRpc2suIFJvbGxpbmcgdGhlIGZpbGUgYmFjayB0byBhdm9pZCBhIG1pc21hdGNoIGJldHdlZW4gZGlzayBhbmQga2VybmVsLiIKICBjcCAtYSAiJHtCQUNLVVBfRElSfS9vcHQvc2NyaXB0cy9pcHRhYmxlcy9zdGFydCIgIiRUQVJHRVQiCiAgZXhpdCAxCmZpCmxvZyBJTkZPICJpcHRhYmxlcyByZWxvYWRlZC4gVmVyaWZ5aW5nIHRoZSBMSVZFIHJ1bGVzZXQgYWN0dWFsbHkgbWF0Y2hlcyB3aGF0IHdhcyBpbnRlbmRlZC4uLiIKCiMgTGl2ZSB2ZXJpZmljYXRpb24gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiMgQ29tcGFyZSB0aGUgcnVubmluZyBrZXJuZWwncyBQUkVMVURFX0VHUkVTU19NQVNRIGNoYWluLCBydWxlLWZvci1ydWxlIGFuZCBpbgojIG9yZGVyLCBhZ2FpbnN0IGV4YWN0bHkgd2hhdCB3ZSBleHBlY3QuIEEgZHJ5LXJ1biB0ZXN0IGNhbm5vdCBjYXRjaCBydW50aW1lCiMgcmVvcmRlcmluZyBieSBvdGhlciByZWNvbmNpbGVyczsgb25seSBpbnNwZWN0aW5nIHRoZSBsaXZlIHRhYmxlIGFmdGVyIGEgcmVhbAojIHJlbG9hZCBjYW4uIFNlZSB0aGUgUkVBRE1FIGZvciB3aHkgdGhpcyBjaGVjayBleGlzdHMuCkVYUEVDVEVEX0NIQUlOPSIkKHByaW50ZiAtLSAnLU4gJXNcbi1BICVzIC1kICVzIC1qIFJFVFVSTlxuLUEgJXMgLWQgMjI0LjAuMC4wLzQgLWogUkVUVVJOXG4tQSAlcyAtaiBNQVNRVUVSQURFIC0tcmFuZG9tLWZ1bGx5JyBcCiAgIiRDSEFJTiIgIiRDSEFJTiIgIiRDQyIgIiRDSEFJTiIgIiRDSEFJTiIpIgpBQ1RVQUxfQ0hBSU49IiQoaXB0YWJsZXMgLXQgbmF0IC1TICIkQ0hBSU4iIDI+L2Rldi9udWxsKSIKCmlmIFsgIiRFWFBFQ1RFRF9DSEFJTiIgIT0gIiRBQ1RVQUxfQ0hBSU4iIF07IHRoZW4KICBsb2cgRVJST1IgIkxpdmUgY2hhaW4gJHtDSEFJTn0gZG9lcyBub3QgbWF0Y2ggd2hhdCB3YXMgZXhwZWN0ZWQgYWZ0ZXIgcmVsb2FkLiIKICBsb2cgRVJST1IgIkV4cGVjdGVkOiIKICBlY2hvICIkRVhQRUNURURfQ0hBSU4iIHwgc2VkICdzL14vICAgIC8nID4mMgogIGxvZyBFUlJPUiAiQWN0dWFsOiIKICBlY2hvICIke0FDVFVBTF9DSEFJTjotPGNoYWluIG5vdCBmb3VuZD59IiB8IHNlZCAncy9eLyAgICAvJyA+JjIKICByb2xsYmFjawogIGV4aXQgMQpmaQoKaWYgISBpcHRhYmxlcyAtdCBuYXQgLUMgUE9TVFJPVVRJTkcgLXMgIiRDQyIgLW8gZXRoMCAtaiAiJENIQUlOIiAyPi9kZXYvbnVsbDsgdGhlbgogIGxvZyBFUlJPUiAiTGl2ZSBQT1NUUk9VVElORyBqdW1wICctcyAke0NDfSAtbyBldGgwIC1qICR7Q0hBSU59JyBpcyBtaXNzaW5nIGFmdGVyIHJlbG9hZC4iCiAgcm9sbGJhY2sKICBleGl0IDEKZmkKCmxvZyBJTkZPICJMaXZlIHZlcmlmaWNhdGlvbiBPSzogJHtDSEFJTn0gZXhpc3RzIHdpdGggZXhhY3RseSB0aGUgZXhwZWN0ZWQgcnVsZXMsIGluIG9yZGVyLCBhbmQiCmxvZyBJTkZPICJQT1NUUk9VVElORyBqdW1wcyB0byBpdCBjb3JyZWN0bHkuIgpsb2cgSU5GTyAiJHtQUn0gY29tcGxldGUgb24gJHtIfS4iCg=="