test me

Site Search:

Understanding Sweat Economy: Realistic Price Modeling and Long-Term Projections

Understanding Sweat Economy: Realistic Price Modeling and Long-Term Projections

The Sweat Economy, a move-to-earn crypto platform, rewards users with $SWEAT tokens for their physical activity. As appealing as it sounds, many users wonder whether these tokens are actually valuable in the long term, especially when step inflation and platform-level burns are factored in. In this post, we model and simulate realistic long-term scenarios for $SWEAT earnings and evaluate whether the Sweat Economy makes sense as an investment of your time and energy.


The Basics of Sweat Economy

  • Sweatcoin App: Users earn non-crypto points (Sweatcoins) by walking.

  • Sweat Wallet App: Converts Sweatcoins into $SWEAT tokens (Web3).

  • $SWEAT Token: Can be staked, traded, or used in the platform's ecosystem.

Initially, users earned 1 $SWEAT for every 1,000 steps. However, the platform introduced step inflation to make it progressively harder to mint new tokens. As of April 2025, you now need over 7,000 steps to earn 1 $SWEAT.


System Architecture Overview

Sweat Economy operates with a modular and scalable architecture designed to support both Web2 and Web3 layers:

Components:

  • Mobile Client (Sweatcoin App + Sweat Wallet): Tracks steps, manages user balances, and interacts with backend services.

  • Step Tracking Service: Integrates with mobile health APIs (Google Fit, Apple Health) and normalizes data.

  • Rewards Engine: Calculates eligible steps, applies rules (e.g. 3,300–10,000 window), and issues Sweatcoins.

  • Conversion Service: Handles user opt-in to Web3 and mints $SWEAT on NEAR Protocol.

  • Smart Contracts: Govern token supply, staking, and burns on the blockchain.

  • Analytics & Pricing Engine: Monitors inflation, adjusts step-to-token ratios, and applies price inflation rules.

Key Design Considerations:

  • Scalability: The backend supports tens of millions of users through horizontal scaling.

  • Security: Blockchain transactions (staking, transfers) are managed via audited smart contracts.

  • Modularity: Allows updates to the step-reward policy, interest models, and gamification logic without disrupting the entire ecosystem.


Modeling Assumptions

To make sense of Sweat Economy, we built a 20-year projection model with the following key assumptions:

Step Earning

  • Only steps between 3,300 and 10,000 per day count.

  • Additional steps come from a daily fortune wheel (4,000 to 10,000 steps).

  • Steps above 10,000 do not earn additional $SWEAT.

Step Inflation (Real Data Based)

  • Start: 7,483 steps per $SWEAT

  • End (1 year later): 15,099 steps per $SWEAT

  • This results in a realistic daily inflation rate of ~0.1925%

Interest Compounding

  • Weekly reinvestment of earned interest into a Sweat "jar"

  • Average annual interest rate of 14.5% based on walking activity (9% to 20% range)

Platform Token Burns

  • Burns occur in years: 2, 5, 8, 12, 16

  • Each burn increases $SWEAT price by 10%

  • Starting token price: $0.00385471 USD


Python Simulation Code

Here is the Python code that performs a realistic 20-year simulation:

import numpy as np
import pandas as pd

# Constants
initial_sweat = 10
initial_token_price = 0.00385471
steps_per_sweat_start = 7483
corrected_daily_inflation_rate = (15099.25 / 7483) ** (1 / 365) - 1
avg_annual_interest_rate = 0.145
burn_years = {2, 5, 8, 12, 16}
price_bump_factor = 1.10
years_to_simulate = 20
days_per_year = 365

# Step range
min_daily_steps = 10000
max_daily_steps = 20000
fortune_min_steps = 4000
fortune_max_steps = 10000

# Initialize
sweat_jar = 0
sweat_balance = initial_sweat
current_steps_per_sweat = steps_per_sweat_start
token_price = initial_token_price
weekly_buffer = 0
days_since_deposit = 0
projection = []

# Simulation loop
for year in range(1, years_to_simulate + 1):
    for day in range(days_per_year):
        daily_steps = np.random.randint(min_daily_steps, max_daily_steps + 1)
        fortune_steps = np.random.randint(fortune_min_steps, fortune_max_steps + 1)
        total_steps = min(daily_steps + fortune_steps, 25000)

        eligible_steps = max(0, min(total_steps, 10000) - 3300)
        earned_sweat = eligible_steps / current_steps_per_sweat
        sweat_balance += earned_sweat

        daily_interest_rate = avg_annual_interest_rate / days_per_year
        weekly_buffer += sweat_jar * daily_interest_rate

        days_since_deposit += 1
        if days_since_deposit == 7:
            sweat_jar += sweat_balance + weekly_buffer
            sweat_balance = 0
            weekly_buffer = 0
            days_since_deposit = 0

        current_steps_per_sweat *= (1 + corrected_daily_inflation_rate)

    if year in burn_years:
        token_price *= price_bump_factor

    projection.append((year, sweat_jar, token_price, sweat_jar * token_price))

# Convert to DataFrame
import matplotlib.pyplot as plt

df = pd.DataFrame(projection, columns=["Year", "Sweat Balance", "Token Price (USD)", "Total Value (USD)"])

plt.figure(figsize=(10, 6))
plt.plot(df["Year"], df["Total Value (USD)"], label="Projected USD Value")
plt.xlabel("Year")
plt.ylabel("USD Value")
plt.title("20-Year Sweat Economy Projection")
plt.grid(True)
plt.legend()
plt.tight_layout()
plt.show()

Results Overview

By Year 5:

  • Sweat Balance: ~794.8

  • Token Price: ~$0.00466 (after two burns)

  • USD Value: ~$3.71

At this point, earning Sweat via walking is approaching 0 due to step inflation, the interest compound is dominant. With weekly reinvestment, a $1 investment grows to approximately $1.65 at 10% and $2.71 at 20% annual interest after 5 years. Over 10 years, these amounts increase to around $2.72 at 10% and $7.36 at 20%.

By Year 20 (projected):

  • Sweat Balance: ~1,900+

  • USD Value: ~$10–12 (depending on future price bumps and activity)


Does Sweat Economy Make Sense?

The Pros:

  • Passive earning for something you already do: walking.

  • Incentivizes daily activity and builds healthy habits.

  • Onboarding millions into Web3 via low-friction mobile app.

The Cons:

  • Token price is low and earnings are limited.

  • Step inflation significantly reduces long-term earning power.

  • Requires consistent walking and staking to make any noticeable gain.

Bottom Line:

Sweat Economy makes sense as a gamified fitness tracker, not as a get-rich-quick scheme. If you’re walking anyway, it’s a nice bonus. But if you expect real financial gain, it's only worth it if you join at the launch time when step inflation is low, or the token gains utility and broader adoption. Buying Sweat with cash and hold for annual compound is risky, if you treat it as a long term investment.


What’s Next?

  • Track real-time token burns and price movement.

  • Wait for more DeFi features and staking utilities.

  • Possibly cash out every few years if the token sees meaningful adoption.

Let us know if you'd like to simulate your own walking pattern or build an app around this ecosystem!

How to Inspect and Clean macOS Network Interfaces (Bridge0, utun, VPN)

How to Inspect macOS Network Interfaces and Clean Up Leftovers

How to Inspect macOS Network Interfaces and Clean Up Leftovers

Have you ever peeked at your Mac's network configuration and wondered what all those mysterious interfaces are? Or perhaps your fan spins up like a jet engine after a fresh reboot, even when no apps are open? If you're curious (or concerned) about what's running under the hood of macOS, this guide is for you.

In this post, we'll explore practical commands to inspect your Mac's networking stack, explain what each interface means, and walk through how to safely clean up unused or leftover configurations. Plus, we'll uncover whether anything fishy is going on with your system.

๐Ÿ’ก Start with the Basics: ifconfig

ifconfig

Example output:

en0: flags=8863<UP,BROADCAST,RUNNING> mtu 1500
    ether aa:bb:cc:dd:ee:ff
    inet 192.168.1.22 netmask 0xffffff00 broadcast 192.168.1.255
    inet6 fe80::1c2b:3eff:fe4a:5a6b%en0 prefixlen 64 scopeid 0x4

Common Interfaces Explained:

  • en0: Your primary network interface (usually Wi-Fi).
  • en1 ~ en4: Often virtual or hardware interfaces (e.g., Thunderbolt, USB Ethernet).
  • bridge0: A virtual interface created by Thunderbolt Bridge or virtualization tools.
  • utun0 ~ utunN: Virtual tunnel interfaces, used by VPNs or macOS services like Handoff.
  • awdl0, llw0: Apple Wireless Direct Link (AirDrop, Handoff).

๐Ÿšซ Thunderbolt Bridge Keeps Coming Back?

macOS Ventura (Darwin 22.6.0) and later automatically re-creates this interface on boot for Thunderbolt-capable Macs.

Fix It:

  • Go to System Settings > Network
  • Select Thunderbolt Bridge
  • Click the ... menu > Make Service Inactive

๐Ÿ›ก️ Investigate Tunnel Interfaces

netstat -nr | grep utun
sudo lsof -i | grep utun

๐Ÿ”Ž Identify Suspicious Listeners

sudo lsof -i -n | grep LISTEN
sudo lsof -p [PID] | grep txt
codesign -dv --verbose=4 /path/to/binary

๐Ÿ” Other Useful Diagnostics

systemextensionsctl list
scutil --nc list
ps aux | grep -i vpn
ls /Library/LaunchAgents
ls /Library/LaunchDaemons
ls ~/Library/LaunchAgents

๐Ÿงน Clean Up VMware, Citrix, and Virtualization Leftovers

Extra interfaces like utun3, utun4, or vmnet can appear due to virtualization tools.

Commands to clean them up:

sudo rm /Library/LaunchDaemons/com.vmware.*
rm -rf ~/Library/Application\ Support/VMware*
sudo launchctl remove com.vmware.CDSHelper
sudo rm -rf /Applications/Citrix\ Workspace.app
sudo rm -rf /Library/LaunchAgents/com.citrix.*
sudo rm -rf /Library/LaunchDaemons/com.citrix.*

Strategy to Identify Leftovers:

sudo find /Library /System/Library /Applications /private -iname '*vmware*' -or -iname '*citrix*'
kextstat | grep -iE 'vmware|citrix|virtual'
launchctl list | grep -iE 'vmware|citrix|vpn'
ps aux | grep -iE 'vmware|citrix|vpn'

๐Ÿ”’ Kernel Extensions Check

kextstat | grep -v com.apple

๐Ÿงผ Trim Unused Network Services

sudo rm /Library/Preferences/SystemConfiguration/NetworkInterfaces.plist
sudo rm /Library/Preferences/SystemConfiguration/preferences.plist
sudo reboot

๐Ÿ’ก Why Is My Fan Spinning Loudly?

Activity Monitor

Try these:

System Settings > Accessibility > Display
[ ] Reduce Transparency
[ ] Reduce Motion

๐Ÿ“Š Final Takeaways

  • bridge0, utunX, awdl0 are usually normal.
  • Ventura recreates Thunderbolt Bridge — make it inactive, don’t delete.
  • Use lsof, codesign, and ps to inspect listeners.
  • Explore launchctl, kextstat, and find for deeper cleanup.
  • Resetting network plists is a clean nuclear fix.

Stay curious, and keep your Mac clean ✨

Exploring Distributed Ledger Platforms and Smart Contracts: A Comparative Guide

Exploring Distributed Ledger Platforms and Smart Contracts: A Comparative Guide

Exploring Distributed Ledger Platforms and Smart Contracts: A Comparative Guide

Welcome to XYZ Network! In today's post, we delve into the world of distributed ledger technology and smart contracts. As blockchain and DLT continue to revolutionize industries, understanding the various platforms and their unique features can help you choose the right tool for your business needs. In this guide, we'll explore several prominent platforms—from DAML/Canton to Ethereum, Hyperledger Fabric, Corda, and beyond—and summarize their key aspects in a comparative table.

What Are Distributed Ledgers and Smart Contracts?

A distributed ledger is a digital record of transactions that is maintained across multiple nodes or computers. This decentralized approach ensures transparency, security, and tamper-resistance, making it ideal for industries such as finance, supply chain, and healthcare. Smart contracts are self-executing contracts with the terms of the agreement directly written into code. They automate processes and enforce contractual obligations without the need for intermediaries.

Bitcoin’s Pioneering Role

Before we dive into platforms with robust smart contract capabilities, it's important to acknowledge Bitcoin's seminal role in the blockchain ecosystem. Bitcoin introduced the concept of a decentralized ledger, proving that a distributed network can securely manage digital transactions without a central authority. Although Bitcoin is primarily a digital currency and store of value with limited smart contract functionality, its pioneering technology laid the groundwork for the development of more advanced platforms like Ethereum and others discussed below.

Overview of Popular Platforms

Each platform has its own design philosophy, consensus mechanism, and target industry. Here’s a quick rundown of some popular platforms:

  • DAML/Canton: Aimed at enterprise solutions, DAML is a smart contract language that, when run on the Canton platform, offers high performance, strong privacy, and interoperability for complex workflows.
  • Ethereum: The pioneer of decentralized smart contracts, Ethereum uses languages like Solidity and Vyper and has built a vast ecosystem of decentralized applications.
  • Hyperledger Fabric: This permissioned blockchain supports private transactions and modular architectures, making it an excellent choice for enterprise use. Smart contracts (or chaincode) here are typically written in Go, JavaScript, or Java.
  • Corda (R3): Specifically designed for the financial sector, Corda emphasizes privacy and direct peer-to-peer interactions, with smart contracts typically developed in Kotlin or Java.
  • Tezos, Cardano, and EOS: These platforms further extend the capabilities of public blockchains with features such as formal verification, academic research foundations, and high throughput.

Comparative Table of Platforms

Platform Ledger Type Smart Contract Language(s) Key Features / Notes
DAML/Canton Permissioned DAML Enterprise-grade; high performance; strong privacy and interoperability; tailored for complex workflows.
Ethereum Public Solidity, Vyper Widely adopted; decentralized; robust developer community; supports a vast ecosystem of dApps.
Hyperledger Fabric Permissioned Go, JavaScript, Java Modular architecture; enterprise-focused; supports private transactions and customizable networks.
Corda (R3) Permissioned Kotlin, Java Designed for financial services; emphasizes privacy and direct peer-to-peer interactions.
Tezos Public Michelson Focuses on formal verification; self-amending blockchain; research-driven design.
Cardano Public Plutus, Marlowe Emphasizes security and sustainability; built on academic research; uses formal methods in its design.
EOS Public C++ Optimized for high throughput; scalable dApps; uses delegated proof-of-stake for consensus.

Why This Matters

Understanding these platforms can help you determine which one aligns best with your project requirements. Whether you're building a decentralized financial application, a supply chain solution, or any other blockchain-based project, knowing the strengths and weaknesses of each platform is crucial. While Bitcoin is not featured in the comparative table due to its limited smart contract functionality, its pioneering role in establishing a decentralized ledger cannot be understated. Bitcoin’s innovation set the stage for the sophisticated platforms we discuss today.

At XYZ Network, we’re committed to bringing you insights into the latest tech innovations. If you have any questions or would like to share your experiences with these platforms, please leave a comment below. Don’t forget to subscribe for more updates on distributed ledger technologies and smart contracts!

Thank you for reading, and stay tuned for our next post on emerging blockchain trends.

Computer networking under the hood

The author created this network study guide by breaking down large subject into small topics, so that we can learn each topic in detail. Computer networking is the foundation of modern society. Here we discuss computer networks built on hardwares, cloud computing, though based on the concepts studied here, is not discussed in this course.

Do It Yourself (DIY) is the best way of learning TCP/IP network, therefore, I highly recommend you to download wireshark and packet tracer. With wireshark, you can capture your network traffic and see exactly what's going on under the hood; with packet tracer, you can set up LAN and WAN virtually and do experiments on them. With some spare bucks, you can even setup a WAN lab at home by purchasing some cisco equipments.

Knowing networking technology is not only fun, it can also get you hired. The most popular network certification is Cisco CCENT and CCNA. If you haven't decided what CCNA test to take, read this post first. Also check out the Cisco device based networking Lab for the labs corresponding to this tutorial.

Whenever you need to learn more about a networking key word, use the Site Search at the upper left conner.

Good luck with your study and test!

(part 1)

Lesson 1
Networking Fundamentals and the Networking Models


In this first lesson, we talk about what is network in general. OSI model and TCP/IP model are visited. These models break network communication into smaller, simpler layers, which make learning easier. Within the context of network layers, the data transmission process is discussed, you will get a bird-view of how the internet makes your web surfing works even at the bit (electromagnetic pulse in cable wires) level. Several famous protocols are introduced: TCP, UDP, IP and ICMP. You will understand the concepts of Ports, Sockets and Port numbers, which are important to software engineers as well as network administrators. Finally, a few common TCP/IP network applications -- FTP, telnet, http are examined with wireshark, so that you get an sense of how these everyday magic happens under the hood.

* The Network We Know about
What is A Network
The TCP/IP Model
The OSI Model
The Data Transmission Process
* Same-Layer and Adjacent-Layer Interactions
TCP And UDP
IP and ICMP protocols
Ports, Sockets and Port Numbers
TCP/IP Applications
Network Topology

Lesson 2
Ethernet Standards and Cable Types


In this foot on the ground lesson, we delve into Ethernet standards and cable types. You will know about connectors such as RJ-11, RJ-45, fiber-optic GBIC and cable types such as UTP, STP, Fiber Optic. You will understand the category of UTP cables, so that when your cable guy says "Your home needs at least Cat 5", you can remain cool. You even gain knowledge about the pins on a cable, able to differentiate a straight-through cable from a crossover cable. Once two computers are connected with a cable, they need to communicate via the electromagnetic pulses in the cable wires. Physicists in the last century took a lot of effort to figured it out. This know how is the foundation of internet. We will learn this know how in topics CSMA/CD, Ethernet frame, ethernet addressing and host-to-host communication. This lesson has a lot of nitty-ditty details, but is crucial for setting up and trouble-shooting a real-world network made of metal and plastics.

What is a LAN
Ethernet Types And Standards
The Need For And Operation of CSMA/CD
Ethernet Frame
Ethernet Addressing
Host-to-Host Communication
Ethernet Connectors and Cable Types
Pins And Transmissions
Crosstalk

Lession 3
Fundamentals of WANs


LAN and WAN are build on OSI layer 1 and layer 2, where cable details such as transmission speeds, encoding, frames and physical links etc. are defined. We have learned LAN before, here lets talk about WAN. Compared with LAN, LAN usually covers a small distance, while WAN spans across a large geographic area. Your party owns LAN, but you most likely share WAN with other parties, so you probably don't have rights to modify the WAN infrastructure.


Lession 4
Switching


Now we know how 2 computers talk to each other, this lesson will study how a group of computers talk to each other. In this lesson we will met layer 1 devices repeaters and hubs, layer 2 devices bridges and switches, layer 3 device routers. While hubs and repeaters allow group computers to share links, switches and bridges divide computer groups in LANs. Routers defines the boundary of LANs.

Ethernet LAN Segments
Repeaters, Hubs, Bridges, Switches and Routers
MAC address learning and filter/forward decisions
Frame Processing Methods
Virtual LANs
Cisco Three-Layer Switching Model
Introduction to STP
Basic Switch Security

Lesson 5
Common Router and Switch Commands


Typical Switch (And Router) Commands are studied here, these commands are used daily by network administrators.

Video Lab - Packet Tracer Interface Overview
Physical Connections and Passwords
Cisco IOS software overview
Physical Side of Cisco Switches
User, Enable and Privilege Modes
Basic Switch managment commands
Switch/Router passwords configurations
Telnet and SSH
Switch Port Security Defaults, Options and Configurations
Banners“logging synch”, and “exec-timeout”
Keystroke Shortcuts and Manipulating History

Lesson 6
IP Addressing and the Routing Process


This lesson covers the fundamental concepts for IP addressing: binary math, subnetting, and working with network and port address translations.

IP Addressing and Binary Conversions
IP Address Classes
Private IP Address Ranges
CIDR network address
Intro to the Routing Process
Routing Process Continued -- Behind the "PING"
* Routing Protocols
Basic Router management commands
Switch/Router Interfaces and Physical Ports

Lesson 7
ARP, DNS and DHCP


DNS Process
The ARP RARP and DHCP Process
Broadcast, Multicast, and Unicast
Intro to Security Device Manager (SDM)

Lesson 8
Memory Components and Config Files


This lesson covers basic password and security configurations, as well as assigning privilege levels, which is the foundation of router security and the basic password recovery process.

ROM, RAM, NVRAM, And Flash
The Boot Process
Setup Mode
Configuration Files and IOS Upgrading
The Configuration Register

Lesson 9
Intro to Wireless Networks (WLANs)


Learn the protocol as well as physics side of wireless.

Wireless LAN overview
Wireless Standards
Spread Spectrum
Antena Types
CA vs. CD
SSID, MAC Address Authentication, WEP, WPA, and WPA2

Lesson 10
Binary Math and Subnetting


The content learned in this lesson is essential for IP addressing and IP address conservation.

Decimal > Binary, Binary > Decimal
Subnetting Basics
Four common subnetting scenarios

Lesson 11
Static Routing and RIP – Part 1


In this lesson, you will see the work done over a Cisco router. You will learn how to manually set up routing.

Static Routing Theory
Floating Static Routing
RIP Routing Theory
Video Lab -Static Routing and RIP Routing

Lesson 12
Wide Area Networks (WANs)


At the end of this course, let's revisit some WAN concepts and get some hands on exercise with cisco equipment

WAN interface of Cisco Router and WAN Cabling
* Layer 2 WAN encapsulations
HDLC and PPP
NAT and PAT
NAT overloading Example
Video Lab - Internet Connections with NAT and PAT
Video Lab - Router as DHCP Server
WAN Trouble-shooting

Lesson 13

Introduction to Network Security


Network security has become more and more important in today's internet. In this lesson, you will learn about network attackers and intruders, how they get in, and how to keep your network save by keeping them out.

The need for network security
Classes of Hacker Attacks
Firewalls and Proxy Servers
The Attacker’s Arsenal
Intro to PIX, ASA, IDS, and IPS
Viruses, Worms, and Trojan Horses
Preventing Virus Attacks

Lesson 14
Troubleshooting


95% of work in the real world is troubleshooting, so it’s necessary for real world success.

Cisco Discovery protocol (CDP)
L1 and L2 Troubleshooting
Telnet and SSH Maintenance Commands
* Administrative Distance
Extended Ping and Traceroute



(part 2)


Lesson 1
Switching II

This lesson revisits the basic concepts of switch and VLAN learned in CCENT, adds more cisco labs about how to configure the switch and router for VLAN.

Switch Basic Concept Review
* STP
* Root Bridges, Root Ports, and Designated Ports
* STP Timers and Port States
* Portfast
* VLANs and Trunking
* Access and Trunk Port Comparison
* VTP
* “Router on a Stick”
* RSTP and PVST
* Etherchannels



Lesson 2
PTP WAN Links, HDLC, PPP, and Frame Relay


This lesson is about Frame Relay.

WAN Trouble-shooting
* HDLC vs. PPP
* WAN Topology
* PPP Features
* PAP and CHAP
* Frame Relay Introduction
* Frame Relay LMI Theory
* Frame Relay Configs, DLCIs, Frame Maps, and Inverse ARP
* Frame Sub-Interfaces3
* Split Horizon
* Frame Relay LMI Show, Debug, and Lab
* FECN, BECN, DE bits
* PVC Status Meanings

Lesson 3
Static Routing and RIP


covering advanced topics about RIP.

* Static Routing Theory and Configuration
* Distance Vector Protocol Behavior – Split Horizon and Route Poisoning
* RIP Theory and Version Differences
* The Joy of “show ip protocols”
* RIP Limitations
* RIP Timers
* Floating Static Routes



Lesson 4
OSPF


OSPF is an Internet protocol we need to know how to configure.

* Link State Routing Protocol Concepts and Basics
* The DR and BDR
* Hello Packets
* Troubleshooting Adjacency Issues
* Hub-and-Spoke NBMA OSPF Networks
* Broadcast Networks
* The OSPF RID
* OSPF Router Types
* Advantages of OSPF
* Point-to-Point OSPF Networks
* Default-Information Originate (always?)
* OSPF Authentication



Lesson 5
EIGRP


this hybrid routing protocol has increased operational efficiency from it predecessor.

* Introduction to EIGRP
* Successors and Feasible Successors
* EIGRP vs. RIPv2
* Basic Configuration
* Wildcard Masks
* Load Sharing (Equal and Unequal-cost)
* EIGRP, RIPv2, and Autosummarization
* Passive vs. Active Routes



Lesson 6
IP Version 6 and NAT


Learn the basic theory and routing protocol of IP version 6.

* IPv6 Theory and Introduction
* Zero Compression and Leading Zero Compression
* IPv6 Reserved Addresses
* The Autoconfiguration Process
* OSPF v3 Basics
* Transition Strategies
* NAT Theory and Introduction
* Static NAT Configuration
* Dynamic NAT Configuration
* PAT Configuration



Lesson 7
VPNs and IPSec


This lesson is about how to setup and run a VPN network.

* Definitions and Tunneling Protocols
* Data Encryption Technologies
* Key Encryption Schemes
* IPSec, AH and ESP
* A VPN in Your Web Browser



Lesson 8
ACLs and Route Summarization


Learn to configure and control ACLs. Learn the basic breakdown and how to summarize routes. Learn common commands for working with RIP & EIGRP.

* ACL Login and the Implicit Deny
* Standard ACLs and Remarks
* “Host” and “Any”
* The Order of the Lines
* Extended ACLs
* Named ACLs
* Telnet Access, Placing ACLs, and Blocking Pings
* Dynamic and Time-Based ACLs
* Port Number Review
* Route Summarization with RIP and EIGRP

See also 640-822 ICND1 Exam Topics (Blueprint)






How spring reactive webclient handle self-signed certificate

Have your spring reactive webclient ever get error:

reactor.core.Exceptions$ReactiveException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
...
Caused by: java.security.cert.CertificateException: No name matching my.host.name.for.post found
              at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:231)

and you have to write code such as the following just to pass your QA integration test?

public Mono<ResponseWrapper> execute(Config conf, MyRequest request) throws SSLException {
    return org.spingframework.web.reactive.function.client.WebClient
                .builder()
                //.clientConnector(getHttpConnector(conf))
                .clientConnector(getQAOnlyConnector(conf)) //TODO: remove this!!!!Don't forget!!!!
                .defaultHeaders(getHeaderConsumer())
                .build()
                .post()
                .uri(getUri())
                .body(BodyInserters.fromValue(request))
                .exchange()
                .flatMap(clientResp -> clientResp.bodyToMono(getResponseClass()))
                .map(respObj -> {
                        ResponseWrapper wrapper = new ResponseWrapper();
                        wrapper.setObj(respObj);
                        return wrapper;
                 });
}

private ReactorClientHttpConnector getQAOnlyConnector(Config conf) throws SSLException {
    SslContext ssl = io.netty.handler.ssol.SslContextBuilder
                   .forClient()
                   .trustManager(io.netty.handler.sso.util.InsecureTrustManagerFactory.INSTANCE)
                   .build();
    HttpClient client = reactor.netty.http.client.HttpClient.create().secure(sslContextSpec -> sslContextSpec.sslContext(ssl);
    return new ReactorClientHttpConnector(client);
}


Well, it is time to get to the bottom of it.

First of all, we need to read the error message, "General SSLEngine problem" means the problem happened during certificate verification process. This exact problem is the hostname.

Let's see what the mismatch is. To rule out the java code error, let's use postman to double check.

The information you need to collect from the java code is the url, headers, post body.

Say, if the postman do get result back, something is "wrong" with your code. Why postman can success but spring reactive http client fails. The reason is, postman by default, don't verify server certificate, however, if you post to a url with https, the spring reactive webclient will try to verify the server's certificate. Some issue is found, and the spring webclient throws.

Ok, let's look into the issue further. If you don't already know, there are two clickable red links at right side above the main request window. One link is "Cookies", another link is "Code". Click that "Code" link, a curl command almost equivalent to the postman request will popup.

Use that curl command, we can look into the issue further.

The curl command could looks like the following:

curl --location --request POST 'https://my.host.name.for.post/someendpoint' \
--header 'Authorization: agiberishstring' \
--header 'Content-Type: txt/xml' \
--data-raw '<Request><id>123</id></Request>'

The curl will complain
curl: (60) SSL certificate problem: self signed certificate
...
If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option.
...
If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option.

The message says it all. The QA server is using a self-signed certificate, no surprise the certificate itself has discrepancies.

Let's reproduce what the postman and the spring reactive client with InsecureTrustManagerFactory's behavior.

The following curl command is the equivalent to the above 2.

curl --insecure --location --request POST 'https://my.host.name.for.post/someendpoint' \
--header 'Authorization: agiberishstring' \
--header 'Content-Type: txt/xml' \
--data-raw '<Request><id>123</id></Request>'

The --insecure flag just tell curl command, don't bother to verify the server's certificate, we don't care its identity or authenticity, just return me the response.
The java code
.trustManager(io.netty.handler.sso.util.InsecureTrustManagerFactory.INSTANCE)
basically convey same message to the spring reactive webclient.

Now let's figure out what happens if the http client (curl or postman or spring webclient) try to verify the server certificate. To do that. Let's supply the curl command the CA certificates.

The reason curl command or spring reactive client want a cacert file is because the CA or (Certificate Authority) file is the root of the trust chain. We may don't know the host's public key, but that public key is signed by someone else, whose public key we trust. The cacert file stores the public keys for all the root CAs we trust.

The trust is established like this: the server's public key is signed by an intermediate CA, whose public key is again signed by another CA,... like a chain..., finally, the chain end at root CA, whose public key is a self assigned certificate. The root CAs are so famous that everybody on the planet earth trust them. For example, root CAs such as verisign, bank of america, etc, they can act as public notary organization to verify other organizations by signing their public key. In other examples, CAs such as your father's company may not good as a public notary organization, but your company can trust your father's company's certificate as one of the root CAs.

Given the curl command a --cacert cacert.pem flag is to tell curl, cacert.pem has all the root CAs I trust, if the target host is eventually signed by any of them, move forward, otherwise stop.

In our case, since the QA server has a self-signed public key, nobody except itself verified its authenticity, we have 3 choices:
  •  trust that self-signed public key as one of the root CA, or 
  • don't trust that self-signed public key, reject it.
  • don't bother to verify the public key at all
Let's trust the self-signed certificate here, and use it as our trusted root CA file. 
To do that, first download CA cert from the server with openssl. The openssl can do that because the server's public key is self-signed, the root CA cert is the server's public key.

echo quit | openssl s_client -showcerts -connect my.host.name.for.post:443 > cacert.pem;

Then use the downloaded cacert.pem as root CA file to verify the server, so that we claim we (blindly) trust that server, want curl command to give it a try.

curl --cacert cacert.pem --location --request POST 'https://my.host.name.for.post/someendpoint' \
--header 'Authorization: agiberishstring' \
--header 'Content-Type: txt/xml' \
--data-raw '<Request><id>123</id></Request>'

This command will fail as the spring reactive webclient does.
curl: (51) SSL: certificate subject name 'localhost' does not match target host name 'my.host.name.for.post'

Great, that is the bottom of it. Even we (blindly) trust that self-signed public key, willing to give it a try, curl or spring webclient is throwing. The programs did a sanity check for the server's public key, find it is not even a correct one. The CN in the certificate is not matching the hostname in target url, which is definitely wrong, so the programs throw.

The operation team who deploy the server machine or VM generated a bad certificate, the host name set in the certificate is "localhost". They might have tested the ssh process on the server, everything is fine, because the test is done on "localhost"!

Let's approve this:

openssl x509 -noout -subject -in cacert.pem
subject= /C=CH/ST=Minst/L=Chill/O=MyOrg/OU=IT/CN=localhost

CN=localhost, but the hostname in our request url is my.host.name.for.post. That is what the spring webclient is complaining about!

If the command has output like this:
openssl x509 -noout -subject -in cacert.pem
subject= /C=CH/ST=Minst/L=Chill/O=MyOrg/OU=IT/CN=myorg.host.name.for.post

the shame is on us, we could have used the correct url in the curl command to make it work.

curl --cacert cacert.pem --location --request POST 'https://myorg.host.name.for.post/someendpoint' \
--header 'Authorization: agiberishstring' \
--header 'Content-Type: txt/xml' \
--data-raw '<Request><id>123</id></Request>'

But for CN=localhost, there is nothing we can do, the server side has to fix it...

Instead of fixing the java code, the right thing to do is to send a polite message to the server maintainer, kindly mention their self-signed certificate has a small problem, please fix it so that we don't have to work around it.

How to ssh to another host via jump hosts

Back>

It is possible to ssh to another host via one or more jumping hosts in the middle, so that the client can act as if the connection were direct.
The main method is to use an ssh connection to forward the ssh protocol through one or more jump hosts using the ProxyJump, to an ssh server running on the target destination host. This method requires the jumpservers enable port forwarding.

ssh -J jumpserver:22 targetserver

In openssh version 7.2 and earlier, passing through jump hosts need the ProxyCommand option to be used either as a run time parameter or as part of ~/.ssh/config.

For example, in order to jump through host jumpserver to host targetserver, we need the following ssh command:

ssh -o ProxyCommand="ssh -W %h:%p jumpserver" targetserver

In this command, the authentication will happen twice, first on the jumpserver, then on the targetserver. So you need to have user/password for both jumpserver and targetserver.

Alternatively, we can put the ProxyCommand as part of ~/.ssh/config

Here is an example ~/.ssh/config

#=================
#~/.ssh/config
#=================
ServerAliveInterval 120

#don't apply any command to ssh localhost
Host localhost
    HostName localhost
    ProxyCommand none

#don't apply any command when ssh to any hostname start with jumpserver
Host jumpserver*
    HostName jumpserver
    ProxyCommand none

#apply proxy jump and ssh connection reuse when ssh to any other hosts
Host *
    ControlMaster auto
    #will create files such as ~/.ssh/master-youruid@targetserver:22 on client host
    ControlPath ~/.ssh/master-%r@%h:%p
    ControlPersist 20m
    ProxyCommand ssh -qA -W %h:%p jumpserver

With the above ~/.ssh/config file, you should be able to ssh into many target server by jumping through jumpserver.

You can forget about the jumpserver and type the normal command:
ssh targetserver

Then you need to enter credential for the jumpserve and targetserver once, then the jumpserver essentially becomes "invisible" in the later ssh connections.

As an extra bonus, the Control* configurations in the ~/.ssh/config prevent you to reentering the credentials again and again after the first time. After your first authenticate with the jumpserver and target host (maybe with 2 factor authentication process), you don't need to re-enter the credentials for reconnecting to the same target servers, unless you closed the ssh connection and didn't reconnect within 20 minutes.

The established connection is persisted in the files specified by the ControlPath for 20 minutes. For the target server, once you opened one ssh network connection, opening more ssh communication sessions with the same server has little resource overhead, you don't need any credential for these extra ssh sessions. The reason is, these new sessions didn't open new ssh connection to the jumpserver nor targetserver, they just have to reuse the existing tcp connection to send extra signals to the network socket using multiplexing. It reduces the load on the jumpserver and target server, also has faster response time.

How to use unix commands to troubleshoot network connection problems

Back>

In the era of internet of things, the skills of trouble-shooting network connectivity became more and more import. Due to its small footage and reliability, linux system are the most popular operation systems on numerous web application servers, docker images, AWS virtual machines, GCP pods, etc.

For example, you got some (pagerduty) alert about the the connection timeout exceptions on one of your web servers, the exception shows the target url http://xyznetwork.blogspot.com/2017/08/xyznetwork-how-to_5.html is not reachable.

DNS lookup

The trouble-shooting start at hostname lookup. You need to know if your dns server is able to solve the hostname part of the url to ip address.

The following command:

nslookup xyznetwork.blogspot.com

will reply you with an ip address or complain that "server can't find xyznetwork.blogspot.com: NXDOMAIN".

nslookup also allows reverse lookup ip address for hostname. As you may already guessed
nslookup 172.217.12.129 won't resolve to xyznetwork.blogspot.com, too many blog urls map to the same ip address, so the ip address won't map to any particular blog url.


dig xyznetwork.blogspot.com

will give you more information about he dns lookup process, including the technical details of the response from the dns servers.

with the trace flag, the dig will reveal the trace log of dns lookup process, including which dns servers were requested and which one of them has the authoritative answer about the ip address.

dig +trace xyznetwork.blogspot.com

dig's flag system also makes it a good scripting command.
For example, the most common dns queries are

  1. A (the IP address), 
  2. TXT (text annotations), 
  3. MX (mail exchanges), 
  4. NS nameservers.
by default, dig performs A query, the following commands will issue other types of queries and the +noall +answer control which part of the information to print to stdout.


dig xyznetwork.blogspot.com MX +noall + answer

=============================================
demo>dig xyznetwork.blogspot.com NS +noall +answer

; <<>> DiG 9.10.6 <<>> xyznetwork.blogspot.com NS +noall +answer
;; global options: +cmd
xyznetwork.blogspot.com. 23 IN CNAME blogspot.l.googleusercontent.com.
demo>dig xyznetwork.blogspot.com MX +noall +answer

; <<>> DiG 9.10.6 <<>> xyznetwork.blogspot.com MX +noall +answer
;; global options: +cmd

xyznetwork.blogspot.com. 2943 IN CNAME blogspot.l.googleusercontent.com.
=============================================

If your DNS servers has no problem of solving the hostname, the next check is to check the ip's reachability.

Routing to the target ip

The simple command ping is the first command we should issue.

ping 172.217.12.129
If the ping replies returned are fast and stable, we at least know the routing from the source ip to the target ip is ok and we don't have firewall dropping the network packets between source ip and target ip.

If the ping didn't go through, there are many possibilities. There is no routing to the ip, firewall is blocking us, the target ip disabled the ping reply, the gateway don't allow ping command to go through, etc. Just mention a few.

As a special note, you can ping the broadcast address to figure out the first hop of the routing process.

ping 255.255.255.255

When ping the broadcast address 255.255.255.255, all the discoverable hosts in your LAN will reply its ip address. One of them could be the network gateway, which is usually your router, one of them is the host you issue the ping command. The rest of them are the other hosts. If you don't want a host to be discovered by its neighbors, you can block the broadcast on the network gateway or configure the host to ignore ping traffic in its firewall.

To know more about the routing, use traceroute command
traceroute 172.217.12.129

The traceroute command will display the route taken by packets across an IP network from your host to the target ip. The ip address the packet traversal will be displayed sequentially. It also shows you how systems are connected to each other, letting you see how your ISP connects to the Internet as well as how the target system is connected. Many routers block traceroute command, making the target system topology invisible to users.

If the ping and traceroute shows there is no route to the target ip, we still can not get conclusion by the results of these 2 commands, since some network nodes might be blocking ICMP port.

However, since our web application previously can connect to the target url, we know for sure that, when everything is working, the http port 80 of the target host must open.

Check port availability

nc -zv xyznetwork.blogspot.com 80

To check if a port is open on a particular host, we can use netcat, the advantage the above command over "telnet xyznetwork.blogspot.com 80" are,

  • telnet command might be disabled, 
  • the nc print the result then exit, so we can scripting it for multiple hosts and ports.
For https connections, the port is 443
issue the following command to check the port https protocol needs:
nc -zv xyznetwork.blogspot.com 443

use openssl we can check the public key of the host server, make sure the target host is what we think it is:
openssl s_client -connect xyznetwork.blogspot.com:443
this command also tested the ssl handshaking process is working between your host and the target host.


At this point, if your dns servers can solve the hostname to target ip address, there is working route from host to the target ip, the port for http or https are open, we have to check application layer.

Check http protocol is working

curl http://xyznetwork.blogspot.com/2017/08/xyznetwork-how-to_5.html

curl https://xyznetwork.blogspot.com/2017/08/xyznetwork-how-to_5.html

Try use curl to issue the http GET/POST command to the target url, if the http webserver application hosted on the target server ip is working, we should get the html code wrapped in http response. In the above example, since the target url is a webpage, GET command is all we need to get the http response back from the webserver.

The curl command displays the plain text html code, that the web browser such as google chrome, firefox used to generate the colorful webpage.

If the curl command can not communicate with the target web server with correct http command (default is GET), headers, protocol, url string, request parameter,  request body etc, then it is time to escalate the issue to the network operation center of your organization.

Your network operation center might reply. Hey, we recently applied new firewall rules, your access to the outside url must go through proxy server, here is the proxy server dev.fancycorpproxy.com, the proxy port is 8080.

curl -x 'dev.fancycorpproxy.com:8080' http://xyznetwork.blogspot.com/2017/08/xyznetwork-how-to_5.html

Then you should try with curl command with proxy, if response come back from xyznetwork.blogspot.com, that could explain the connectivity issue. If the proxy server gives you something like 403 forbidden, please contact fancycorp IT administrator at email blabla, they need to add a new firewall rule or a new proxy ACL.

If your network operation center don't have explanation,  In this case it is google.com...probably you won't get to this problem and probably the other side already known about the issue.

Check your own application

Assuming the application logged the connection timeout is a java application, we need to inspect the network connectivity of the process reporting the issue.

netstat -nulpt | grep java

The netstat command will list all the listening port for a process with java in the name. You can figure out if there are established connections to the target server, if the debug port is opening, or someone is currently connecting to the process via a local connection, which indicates the existence of a reverse proxy setup on the host etc.

If you are worrying about rouge host in your network, a tell-tell check is to use arp -a, this list gives away all the hosts you recently connected to. Do the ips you are connecting have the correct MAC address it suppose to be?

demo>arp -a
openrg.home (192.168.1.1) at f6:4f:5a:4:7b:f2 on en1 ifscope [ethernet]
? (224.0.0.251) at 1:0:5e:0:0:fb on en1 ifscope permanent [ethernet]
? (239.255.255.250) at 1:0:5e:7f:ff:fa on en1 ifscope permanent [ethernet]

In the above example, all the 3 entries are normal:


  • 192.168.1.1 is the gateway. 
  • 224.0.0.251 is the address for the multicast DNS (mDNS) protocol. The mDNS protocol resolves hostnames to IP addresses within small networks that do not include a local name server. It is a zero-configuration service, using essentially the same programming interfaces, packet formats and operating semantics as the unicast Domain Name System (DNS). 
  • 239.255.255.250 This address is used for UPnP (Universal Plug and Play)/SSDP (Simple Service Discovery Protocol) by various vendors to advertise the capabilities of (or discover) devices on a VLAN. MAC OS, Microsoft Windows, IOS and other operating systems and applications use this protocol. Client devices can use this protocol to advertise its capabilities to other devices.


How to use dig +trace to reveal DNS lookup internal

Back>

You might already know that dig can be used to solve domain name.
However, you might not know how the dns servers solve the domain name.

Just by simply add +trace flag, the dns resolving process will be revealed.

For example, the DNS lookup starts at the root level dns servers ., and continues from right to left, com. level servers are then queried, then lower level dns servers are queried, finally a DNS server is able to provide an authoritative A record.

For example, xxxxxx.blogspot.com domain names are all resolved to the same ip address, that ip address is one of public ip of google. Behind that ip, web servers dispatch the requests to a particular blog according to the parts before .blogspot.com. In other words, your blog don't have a dedicated ip address, which is good, in the sense of web server security, because google is managing the web server for you so that you just focus on contents.

demo>dig +trace xyzcode.blogspot.com

; <<>> DiG 9.10.6 <<>> +trace xyzcode.blogspot.com
;; global options: +cmd
. 220522 IN NS g.root-servers.net.
. 220522 IN NS h.root-servers.net.
. 220522 IN NS a.root-servers.net.
. 220522 IN NS l.root-servers.net.
. 220522 IN NS k.root-servers.net.
. 220522 IN NS b.root-servers.net.
. 220522 IN NS f.root-servers.net.
. 220522 IN NS d.root-servers.net.
. 220522 IN NS m.root-servers.net.
. 220522 IN NS e.root-servers.net.
. 220522 IN NS c.root-servers.net.
. 220522 IN NS j.root-servers.net.
. 220522 IN NS i.root-servers.net.
;; Received 811 bytes from 68.105.28.11#53(68.105.28.11) in 14 ms

com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS d.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS f.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 86400 IN DS 30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com. 86400 IN RRSIG DS 8 1 86400 20200521170000 20200508160000 48903 . BemkQJ+5wV2uHyc1V/SzRxJKt9GfVupkuDq2TqFY9Kt0tsvaKC6OZp+Y WZuBPZ+qHOU59o3APTBgtBbpDwTH+bXXYrqU3RNutirrwA/Z9RW+J3Bx W771zw5at79UWcZBkq2LxAYW2e3ZVukbQtylm5Wa5TeaBKsfr471dtEP hStNZ1vFrJ7VRt/txo399pn5HIslwuXDDc7LI65Dc8mFxHzjv8f/COQX mOPLESd5QVVd9oatek2lC43ArqI8x6aohLLyXdcSCdm0mVWmaC+4lpzl 3NGwP7GOmRVnuGjFxTZFDPTILYHNTziDPDriEYrNWwGxrHteAA+QB9i4 MdP/kA==
;; Received 1180 bytes from 198.41.0.4#53(a.root-servers.net) in 26 ms

blogspot.com. 172800 IN NS ns2.google.com.
blogspot.com. 172800 IN NS ns1.google.com.
blogspot.com. 172800 IN NS ns3.google.com.
blogspot.com. 172800 IN NS ns4.google.com.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A  NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20200513044951 20200506033951 39844 com. DQ9LaY7nv4abiSkEn0gpiP0cQ8J7yqT4l29DPEUyTure4dT/cQOGGhB4 YaB6r/2IAy0Q32WN2JIPrBQZWYFans5vdqZKOE0bT5WIOCK3TFqfmpKy wcaRIcAqloo2ucXB5WSk30r4+ep3DgkfgQyAmgDfJWM0jMEMPxRYhm3l DBVkbvRe4un6nc1i07mz7d1i25O8nmx24r929EcMKPlF4w==
7E75D4UKK0QJCF521ERANMKGUOOD8KFM.com. 86400 IN NSEC3 1 1 0 - 7E75QJA0KQJU8DPN58K6SB69223LRR8I  NS DS RRSIG
7E75D4UKK0QJCF521ERANMKGUOOD8KFM.com. 86400 IN RRSIG NSEC3 8 2 86400 20200514045819 20200507034819 39844 com. GzCoHQxYKhuO56zahxwBj0Nkp23OwKNjSwyfyvFHvu/QLqeBHTl0uWJT IBBcsBW4lHtfv1arTf73ASrE9x/el1aGZpt2rWe3TIeN73OB70xWFT5G g8T30AP5PxsaJoBJTam2rY8VMpQzVdyd3NC0JqWBZLl7kmqWte2gY9Oa Oh3raD8YKH5kvAQ/tuJt+f6fBrkkXqg2y/mi1F+1jl1l5A==
;; Received 853 bytes from 2001:500:856e::30#53(d.gtld-servers.net) in 63 ms

xyzcode.blogspot.com. 3600 IN CNAME blogspot.l.googleusercontent.com.
blogspot.l.googleusercontent.com. 300 IN A 172.217.12.129
;; Received 108 bytes from 216.239.38.10#53(ns4.google.com) in 37 ms

demo>dig +trace xyzcodexxxxxxxxxxx.blogspot.com

; <<>> DiG 9.10.6 <<>> +trace xyzcodexxxxxxxxxxx.blogspot.com
;; global options: +cmd
. 226599 IN NS m.root-servers.net.
. 226599 IN NS a.root-servers.net.
. 226599 IN NS b.root-servers.net.
. 226599 IN NS c.root-servers.net.
. 226599 IN NS d.root-servers.net.
. 226599 IN NS e.root-servers.net.
. 226599 IN NS f.root-servers.net.
. 226599 IN NS g.root-servers.net.
. 226599 IN NS h.root-servers.net.
. 226599 IN NS i.root-servers.net.
. 226599 IN NS j.root-servers.net.
. 226599 IN NS k.root-servers.net.
. 226599 IN NS l.root-servers.net.
;; Received 811 bytes from 68.105.28.11#53(68.105.28.11) in 12 ms

com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS f.gtld-servers.net.
com. 172800 IN NS d.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 86400 IN DS 30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com. 86400 IN RRSIG DS 8 1 86400 20200521170000 20200508160000 48903 . BemkQJ+5wV2uHyc1V/SzRxJKt9GfVupkuDq2TqFY9Kt0tsvaKC6OZp+Y WZuBPZ+qHOU59o3APTBgtBbpDwTH+bXXYrqU3RNutirrwA/Z9RW+J3Bx W771zw5at79UWcZBkq2LxAYW2e3ZVukbQtylm5Wa5TeaBKsfr471dtEP hStNZ1vFrJ7VRt/txo399pn5HIslwuXDDc7LI65Dc8mFxHzjv8f/COQX mOPLESd5QVVd9oatek2lC43ArqI8x6aohLLyXdcSCdm0mVWmaC+4lpzl 3NGwP7GOmRVnuGjFxTZFDPTILYHNTziDPDriEYrNWwGxrHteAA+QB9i4 MdP/kA==
;; Received 1191 bytes from 2001:500:2::c#53(c.root-servers.net) in 23 ms

blogspot.com. 172800 IN NS ns2.google.com.
blogspot.com. 172800 IN NS ns1.google.com.
blogspot.com. 172800 IN NS ns3.google.com.
blogspot.com. 172800 IN NS ns4.google.com.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A  NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20200513044951 20200506033951 39844 com. DQ9LaY7nv4abiSkEn0gpiP0cQ8J7yqT4l29DPEUyTure4dT/cQOGGhB4 YaB6r/2IAy0Q32WN2JIPrBQZWYFans5vdqZKOE0bT5WIOCK3TFqfmpKy wcaRIcAqloo2ucXB5WSk30r4+ep3DgkfgQyAmgDfJWM0jMEMPxRYhm3l DBVkbvRe4un6nc1i07mz7d1i25O8nmx24r929EcMKPlF4w==
7E75D4UKK0QJCF521ERANMKGUOOD8KFM.com. 86400 IN NSEC3 1 1 0 - 7E75QJA0KQJU8DPN58K6SB69223LRR8I  NS DS RRSIG
7E75D4UKK0QJCF521ERANMKGUOOD8KFM.com. 86400 IN RRSIG NSEC3 8 2 86400 20200514045819 20200507034819 39844 com. GzCoHQxYKhuO56zahxwBj0Nkp23OwKNjSwyfyvFHvu/QLqeBHTl0uWJT IBBcsBW4lHtfv1arTf73ASrE9x/el1aGZpt2rWe3TIeN73OB70xWFT5G g8T30AP5PxsaJoBJTam2rY8VMpQzVdyd3NC0JqWBZLl7kmqWte2gY9Oa Oh3raD8YKH5kvAQ/tuJt+f6fBrkkXqg2y/mi1F+1jl1l5A==
;; Received 864 bytes from 192.43.172.30#53(i.gtld-servers.net) in 28 ms

xyzcodexxxxxxxxxxx.blogspot.com. 3600 IN CNAME blogspot.l.googleusercontent.com.
blogspot.l.googleusercontent.com. 300 IN A 172.217.12.129
;; Received 119 bytes from 216.239.34.10#53(ns2.google.com) in 39 ms

demo>

How to inventory cloud assets in aws cloud

Back>

Cloud assets have an important difference comparing to physical assets. While physical hosts are permanent, the EC2 instances are temporary.

When we make an inventory for the physical network assets, we want to know these information:

cloudtrail-cloudwatch-kinesis
cloudtrail-cloudwatch-kinesis

    • what's the current inventory snapshot.
    • for each host, what is the hostname, ip address, device type, operation system, owner, etc.
    • when a host is added or removed from the inventory.
    So we need to store a set of versioned inventory snapshots. The inventory increases the version when some hosts change at a particular timestamp.

    If we apply the same strategy to inventory aws assets, it most likely won't work. First of all, the aws assets changes too often, the ASG groups are adding and deleting EC2 instances all the time, you need to store tones of snapshots for these changing history. Even though you can catch up with the constant changing cloud, why do you bother to store the hostname and ip address of an EC2 instance that only lived 1 minute in an ASG? We should be more concerned about long lived items such as Elastic load balancer, ASG, tagged assets, etc. We should store hostname, ip address, device type, operation system, owner, group member count, owner, availability zone etc. 

    How do we know if a concerned item changes, we don't want to know the change of every EC2 instances for sure. One solution is to take snapshots periodically, only store a new version when the new snapshot is different than the current one. Another solution is to have the cloudtrail inform us about these changes, then we make a new snapshot. We can configure the cloudtrail to send events to CloudWatch Logs. CloudTrail supports sending data, Insights, and management events to CloudWatch Logs. An event such as "eventName": "CreateLoadBalancer" should trigger a new inventory snapshot to be made. We can have the cloudwatch logs consumed by a Lamda function, then triggered a message to be send to SQS. Then the listening application can take action accordingly. A more sophisticated approach is to have a java/python application call the cloudwatch api for the logs. The application then processes these logs to store information and create events. One of the events is -- the assets we concerned with just changed, our program should submit a job to the job queue for making a new inventory snapshot.

    The inventory information can be retrieved by describing the ELBs, ASGs and EC2s with tag as the filter. A worker thread can pick up the inventory snapshot job, have a new snapshot file created and ETLed to the blobstore.

    This is a basic setup. If you have multiple aws accounts across multiple regions, you need to iterate through the ARNs and regions in order to check the cloudwatch logs or make the inventories, then merge them into one snapshot. You can also configure the cloudwatch logs in different aws accounts to be consumed by to the Kinesis streams in one account. Kinesis streams are currently the only resource supported as a destination for cross-account subscriptions. Then the Kinesis shards can be downloaded in parallel for the logs. Setup the cloudtrail and cloudwatch in customers' aws account don't have to be manual. We can have the program assume across account role, then call  cloudformation api to have the trail and cloudwatch created then subscribed to the cloudwatch destination of the kinesis stream in primary account, if the cloudwatch haven't been found there.

    How to use robot framework to manage a virtual QA lab in aws cloud

    Back>

    Aws cloud makes QA test easier. We can setup the production VPC, then simulate that VPC network in another aws VPC for QA purpose. The word "simulate" is under-estimating the similarity between prod env and QA env, because using the production's cloudformation template, we can build a QA network exactly as the production network. Even better, once we are done with the QA test, we can delete all the assets created by the cloudformation template, and we don't have to pay the maintenance cost for QA lab. Of course, it is a good practice to keep the prod cloudformation template and QA cloudformation template separate, have template variables to make the templates flexible, also divide a big cloudformation template into smaller cloudformation template as modules. Then we don't have to create the whole production network, we can just create part of it for testing.

    Since setup a aws QA network is so easy, we can setup a few cron jobs to run a few QA test projects in parallel. A test project is like a movie theater -- at different times, a cron job put different movies on the show. A show is handled by a robot in robot framework.

    Upon starting by the cron job, the robot will accomplish the following task during its life cycle:
    1. create the QA test network using cloudformation api, the needed cloudformation template can be stored in a S3 bucket.
    2. Once the QA test network is created, the robot will start to run a few test suits.
    3. Assume the robot framework is integrated with grafana, we can then see the test statistics graphically in a dashboard in real time.
    4. After all the test suits are finished, the robot call cloudformation api to delete all the aws assets during the test.
    Now we know the game rules, let's take a look at an example.

    aws QA lab example
    aws QA lab example


    Our example test project is to verify the alert system can escalate a network intrusion event into a ticket.

    The network includes two part: the customer's network and the security center's network. In the customer's network, we have an intrusion detector. The detector will collect logs from the customer's assets. The detector uses a set of regular expression rules to match logs in order to filter out those relevant to network intrusion. (snort for example, detect intrusion this way). Once the regular expression matches something, an event is generated for closer look. The event should at least have the timestamp, sourceIp, destinationIp, description, raw log. These events are send to the event engine located in security center network via a VPN tunnel. On that server, the event is inserted into a mysql database, then the event engine will escalate the relevant event to ticket for human inspection. The ticket should at least have severity, POC, relevant event lists. Once the ticket is generated, the ticket will be ETL to a security analysis's' ticketing system and inserted into their mysql database. The ticket should at least have severity, POC, relevant event lists and status.

    So the pass criteria is:

    1. found a new event in event engine's mysql database, with matching sourceIp, destinationIp, description.
    2. found a new ticket in event engine's mysql database, with expected severity, POC, event lists.
    3. found a new ticket in security analysis's' mysql database with expected severity, POC, event lists and status.
    With these in mind we can have the robot issue a cloudformation api call with parameters to trigger a cloudformation build. The orchestration is achieved with a SQS, which the template will create first.  The template will create an EC2 for the intrusion detector from an existing image,  the EC2 instance needs a secret to call back security center, which will be supplied as an EC2 instance variable. Once the EC2 is created, a startup script will send a message to the SQS, informing its ip address. At the same time, the cloudformation template is also creating 2 EC2 instances as attacker and victim. Once The victim instance is created, a startup script will poll on the SQS for the message with intrusion detector's ip. Once that message is retrieved, the victim EC2 will setup the syslog server to point to the intrusion detector. Then the victim will send a SQS message with its own ip address. The attacker EC2 is polling the SQS for the victim's ip address. Once that message is retrieved, it will create a robot that attacking the victim. Assume the robot issues 3 failed ssh to the victim in order to have the victim send a syslog to intrusion detector reporting this event. (We can have the victim do real hack with hacking tools installed there, but let's assume failed ssh login is enough for our test.) The robot starting the cloudformation template will sleep long enough for the system to fully digest this attacking event. Once it wakes up, it will check the mysql databases for the passing criteria. If no event is found, it will sleep longer and check again. If still not found event, the test will be marked as a failure.

    There are some details not mentioning, the attacker and victim's EC2 are two simple unix OS, with rpms (such as robot framework) downloaded from S3 buckets. The intrusion detector have a http client to call back to the server located in the security center for sending events, the secrete to identify itself is send to the server via vpn tunnel during hand-shake process. This intrusion detector EC2 can also download rpm from S3 bucket. The rpms stored in the S3 bucket is automatically updated by another robot 24/7 through Nexsus repository download and aws S3 api calls.