Refining the video caption to include the development disclaimer and adding the image with the quote. Locating the current video caption:

Updating the video caption with the disclaimer and adding the image below:

  
  
  
  
  


  

    

      ← Back to Home
    


      

        
      
      

        
Week 11
        
Networking & Communications
      
      
    


    

      

        
        Week Highlights
      
      
Key captures from the group and individual assignments with links to their sections.


      

        
        

          
ESP-NOW Multiplayer Network
          
            
          
        
        
        

          
Wi-Fi Camera Livestream
          
            
          
        
        
        

          
CircleSquareCamera AR
          
            
          
        
      
    


    

      

        
        Table of Contents
      
      

        

          
Course Content
          

            
• Week Highlights
            
• Intro
            
• This Week's Goals
            
• Assignments
            
• Tools & Materials
          
        
        

          
Training & Documentation
          

            
• Training Documentation
            
• Recitation Notes
            
• Useful Documentation
            
• Networking Protocols
          
        
        

          
Assignments & Projects
          

            
• Group Assignment
            
• Individual Assignment
            
• Design Files
            
• Reflections & Learnings
            
• Contributions
            
• Ethical AI Use
          
        
      
    


    

      

        
Networking & Communications
        
This week focuses on connecting devices through wired and wireless networks. We'll explore communication protocols, network addressing, and how to build networked systems with local input and output devices.
        

          
Understanding wired and wireless communication protocols
          
Learning network addressing and bus architectures
          
Implementing networked nodes with input and output devices
        
      
      

        
This Week's Goals
        

          
Network Protocols
Learn about ESP-NOW, Wi-Fi, I²C, SPI, UART, and other communication protocols
          
Network Addressing
Understand MAC addresses, IP addresses, and bus addressing schemes
          
Networked Systems
Design, build, and connect wired or wireless nodes with local I/O devices
        
      
    


    

      

        
Assignments
        

          
Group Assignment
Send a message between two projects
          
Individual Assignment
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s)
        
      
      

        
Tools & Materials
        

          
ESP32-S3 Microcontroller Boards
          
Wi-Fi Access Point
          
OLED Displays (SSD1306)
          
Camera Modules
          
Touch Sensors
        
      
    


    

      
Training Documentation
      
Training materials and documentation for networking and communications protocols.


      

        
Recitation Notes
        

          Networking and communications recitation covered ESP-NOW protocol implementation, MAC address-based device identification, peer-to-peer communication setup, and network address configuration for embedded systems.
        
      


      

        
Networking Protocols Overview
        

          Networking and communications enable devices to exchange data through wired and wireless connections. Understanding different protocols helps select the right communication method for each application.
        

Wired Communication
        

          
UART/USART — Asynchronous serial communication (RS-232, RS-422, RS-485)
          
I²C/TWI — Two-wire interface with SCL/SDA, requires pull-up resistors
          
SPI — Serial Peripheral Interface with MISO/MOSI/CS/SCK lines
          
USB — Universal Serial Bus for device communication
          
Ethernet — Wired network communication
        

Wireless Communication
        

          
ESP-NOW — Low-power peer-to-peer communication for ESP32 devices
          
Wi-Fi — 802.11 wireless networking (2.4 GHz, 5.8 GHz)
          
Bluetooth/BLE — Short-range wireless communication
          
LoRa — Long-range, low-power wireless communication
          
RF — Radio frequency communication (433 MHz, 868 MHz, 915 MHz, 2.4 GHz)
        

Network Addressing
        

          Devices on networks require unique addresses for identification and communication:
        
        

          
MAC Address — Media Access Control address, unique hardware identifier (e.g., D8:3B:DA:75:05:AC)
          
IP Address — Internet Protocol address for network routing (e.g., 192.168.1.100)
          
Bus Address — Device address on shared communication buses (I²C, SPI)
        
      
    


    

      
Useful Documentation
      
Essential resources for networking and communications protocols.


        

          
Class Page
          
Networking and Communications - MIT Academy
          
Comprehensive resource covering wired protocols (UART, I²C, SPI, USB, Ethernet), wireless protocols (Wi-Fi, Bluetooth, LoRa, RF), network addressing, OSI layers, modulation techniques, channel sharing methods, error detection/correction, and Internet protocols. Includes detailed tutorials, code examples, and implementation guides for various communication protocols.
        
        

          
Key Topics
          

            
Wired Communication: UART, I²C, SPI, USB, Ethernet, CAN, LIN, MODBUS
            
Wireless Communication: ESP-NOW, Wi-Fi (802.11), Bluetooth/BLE, LoRa, RF (433/868/915 MHz, 2.4/5.8 GHz)
            
Network Protocols: IPv4/IPv6, TCP/UDP, HTTP, DNS, DHCP, NAT
            
Modulation: PCM, PPM, OOK, FSK, BPSK, QAM, OFDM, FHSS, DSSS, UWB
            
Channel Sharing: TDMA, FDMA, CSMA/CD, CSMA/CA, CDMA, MIMO
          
        
      
    


    

      
Group Assignment: Send a Message Between Two Projects
      
Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols.


        
Assignment Description
        

          Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols. The CircleSquareCamera Augmented Reality system extends the ESP-NOW multiplayer game from Week 2 by integrating camera boards that receive game state updates and overlay player positions on live camera feeds.
        
      


      

        

          
          CircleSquareCamera Augmented Reality
        


          An augmented reality system that combines the ESP-NOW multiplayer game CircleSquareShapeSongSwingalong (documented in Week 2 and Week 11 individual assignment) with camera subsystems from the final project. The system enables game boards to send player state updates via ESP-NOW to camera boards, which overlay game characters (square and circle) on live camera feeds displayed on OLED screens. Camera code development spans Week 7, Week 8, Final Project, and Week 12 (placeholder link).
        


          

            
            
          
          

            

              Development Note: Initial implementation attempted to send camera data to game boards, but after troubleshooting communication issues, the architecture was reversed. The final system sends game state from game boards to camera boards via ESP-NOW, which proved more efficient for real-time AR overlay visualization.
            
          
          

            
              
            
            
          
        

Design & Build
        

          The system consists of two networked subsystems: game boards (XIAO ESP32-S3 with touch sensors and OLED displays) and camera boards (XIAO ESP32-S3 Sense with integrated cameras and OLED displays). Game boards run the CircleSquareShapeSongSwingalong multiplayer game, processing touch input, updating player physics, and broadcasting player state packets via ESP-NOW to both the other game board and all camera boards. Camera boards receive these packets, capture camera frames, process them through Floyd–Steinberg dithering for OLED display, and overlay game character positions (square for Player 1, circle for Player 2) on the live feed.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification for game boards and camera boards

          Communication Pattern: One-to-many broadcast from game boards to peer game board and all camera boards
        

Input & Output Devices
        

          
Game Boards:
            

              
Input: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering
            
          
          
Camera Boards:
            

              
Input: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, ESP-NOW packets for game state
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for displaying dithered camera feed with AR overlay
            
          
        

Connections
        

          Game Boards: Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. ESP-NOW communication is handled through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.


          Camera Boards: The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. ESP-NOW reception uses the ESP32-S3's built-in Wi-Fi radio in receive-only mode.
        


        

          
Network Address Tables
          

            Complete network addressing information for all devices in the CircleSquareCamera AR system, including MAC addresses (hardware identifiers) for ESP-NOW communication.
          

ESP-NOW Network (CircleSquareCamera AR System)
          

            

              

                
Device
                
Component
                
MAC Address
                
Network Type
                
Input Device
                
Output Device
              
            
            

              

                
P1
                
XIAO ESP32-S3
                
D8:3B:DA:75:05:AC
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
P2
                
XIAO ESP32-S3
                
D8:3B:DA:75:E1:9C
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
Camera A
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:E2:C0
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
              

                
Camera B
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:D6:38
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
            
          


            

              Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. Game boards broadcast player state packets to both the peer game board and all camera boards. Camera boards operate in receive-only mode, processing incoming ESP-NOW packets to identify player positions and overlay them on camera feeds.
            
          
        


          
Download Design Files
          

            Complete Arduino code for game boards and camera boards, including ESP-NOW communication, camera feed processing, and AR overlay implementation. See Design Files section for detailed documentation and download links.
          
        
      
    


    

      
Individual Assignment: Networking & Communications
      
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s). This assignment demonstrates two networked systems: an ESP-NOW multiplayer game and a Wi-Fi camera streaming system with edge AI processing.


        
Related Documentation
        

          
Week 2: Embedded Programming & Nintendo Training — ESP-NOW multiplayer game development
          
Week 2: Design Files — CircleSquareShapeSongSwingalong Arduino code
          
Week 10: Camera & Edge AI — Wi-Fi camera streaming system
          
Week 10: Design Files - Camera — Camera stream code and system implementation
        
      


      

        
Networked System 1: CircleSquareShapeSongSwingalong (ESP-NOW)
        

          A collaborative multiplayer game featuring geometric shape interactions, synchronized gameplay between two ESP32-S3 devices using ESP-NOW wireless communication. The system uses MAC address-based player identification and real-time state synchronization for coordinated gameplay. See Week 2 documentation and design files for complete details.
        

Design & Build
        

          The system consists of two XIAO ESP32-S3 development boards, each with an SSD1306 OLED display (128x64) and capacitive touch sensors. The boards communicate wirelessly using ESP-NOW protocol, enabling low-latency peer-to-peer communication without requiring a Wi-Fi access point.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification
        

Input & Output Devices
        

          
Input Devices: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
          
Output Devices: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering and player state visualization
        


          

            

              
            
            
Shape Song Swing Along educational content integration and gameplay
          
          

            

              
            
            
Circle and square shape interaction gameplay mechanics
          
        

Connections
        

          Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. The ESP-NOW communication is handled entirely through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.
        


          
Download Design Files
          

            
TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino — Complete Arduino sketch for ESP-NOW multiplayer game
            
get_mac_address.ino — Utility to read MAC addresses from ESP32-S3 devices
          
        
      


      

        
Networked System 2: Camera Livestream & Edge AI (Wi-Fi)
        

          A Wi-Fi-enabled camera system using ESP32-S3's built-in camera and HTTP server capabilities to stream JPEG frames over Wi-Fi using MJPEG (Motion JPEG) protocol. The system also includes edge AI face detection using a FOMO (Faster Objects, More Objects) model from Edge Impulse for real-time person tracking. See Week 10 documentation and design files for complete details.
        

Design & Build
        

          The system uses a XIAO ESP32-S3 Sense development board with an integrated camera module. The camera captures frames at QQVGA (160x120) or QVGA resolution, processes them through an Edge AI inference pipeline, and streams the results over Wi-Fi via HTTP multipart response. The system can also display camera output and inference results on an OLED display connected via I²C.
        

Network Type & Protocol
        

          Network Type: Wireless local area network (Wi-Fi)

          Protocol: HTTP over Wi-Fi (802.11), MJPEG streaming

          Addressing Scheme: IP address (DHCP-assigned) and MAC address (hardware identifier)
        

Input & Output Devices
        

          
Input Devices: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, button for capture trigger
          
Output Devices: OLED display (SSD1306, 128x64, I²C address 0x3C) for displaying camera frames and inference results, Wi-Fi HTTP server for remote streaming
        


          

            
            
          
        
        

          

            
            
          
          

            
            
          
          

            
            
          
        

Connections
        

          The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. Wi-Fi connection is established through the ESP32-S3's built-in Wi-Fi radio.
        


          
Download Design Files
          

            
camera_stream.zip — Complete camera stream project (includes .ino and .h files)
            
camera_stream.ino — Main Arduino sketch for camera livestreaming
            
camera_pins.h — GPIO pin definitions for XIAO ESP32-S3 camera module
            
ei-face-detection--fomo-arduino-1.0.90.zip — Edge Impulse Arduino library for FOMO-based face detection
          
        
      


      

        
Network Address Tables
        

          Complete network addressing information for all connected devices, including MAC addresses (hardware identifiers) and IP addresses (network identifiers) for Wi-Fi-connected devices.
        

ESP-NOW Network (CircleSquareShapeSongSwingalong)
        

          

            

              
Device
              
Component
              
MAC Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Player 1
              
XIAO ESP32-S3
              
D8:3B:DA:75:05:AC
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
            

              
Player 2
              
XIAO ESP32-S3
              
D8:3B:DA:75:E1:9C
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
          
        

Wi-Fi Network (Camera Livestream System)
        

          

            

              
Device
              
Component
              
MAC Address
              
IP Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Camera Node
              
XIAO ESP32-S3 Sense
              
[Device MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
Camera module, Button
              
OLED (SSD1306, 0x3C), HTTP Server
            
            

              
Client Device
              
Web Browser
              
[Client MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
N/A
              
Display (receives MJPEG stream)
            
          
        


          

            Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. IP addresses are assigned dynamically via DHCP when connecting to a Wi-Fi network. For ESP-NOW communication, only MAC addresses are used (no IP addresses required). The camera system's IP address is printed to the Serial Monitor upon Wi-Fi connection establishment.
          
        
      
    


    

      
Design Files
      
Complete design files for networked systems including ESP-NOW multiplayer game and Wi-Fi camera streaming implementations with Arduino firmware and configuration files.


      

        
ESP-NOW Multiplayer Game (CircleSquareShapeSongSwingalong)
        

          Arduino code for ESP-NOW wireless multiplayer game with MAC address-based player identification and synchronized game state management.
        
        

          
Key Features:
          

            
ESP-NOW wireless peer-to-peer communication
            
MAC address-based player identification (D8:3B:DA:75:05:AC and D8:3B:DA:75:E1:9C)
            
Real-time game state synchronization
            
Ready state coordination between players
            
Touch-based input controls (left, right, jump)
            
OLED display output for game rendering
          
        


          

            
              
              View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino
            
          
          

            
              
              Download .ino
            
          
          

            
              
              Download get_mac_address.ino
            
          
        
      


      

        
Wi-Fi Camera Livestream & Edge AI
        

          Arduino code for ESP32-S3 camera livestreaming over Wi-Fi using MJPEG protocol and Edge AI face detection using FOMO models from Edge Impulse.
        


          
Camera Livestream Pseudocode:
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Configure camera pins (from camera_pins.h)
  3. Create camera_config_t structure
  4. Initialize camera with esp_camera_init()
  5. Connect to Wi-Fi network
  6. Start HTTP server with stream handler

STREAM_HANDLER:
  1. Set HTTP response type to "multipart/x-mixed-replace"
  2. Enter loop: capture frame, send via HTTP, repeat

LOOP:
  - Minimal delay to allow other tasks
        



          
Key Features:
          

            
Wi-Fi HTTP server for MJPEG streaming
            
Edge AI face detection using FOMO models
            
Real-time camera frame capture and processing
            
OLED display output for local visualization
            
IP address assignment via DHCP
            
MAC address for hardware identification
          
        


          

            
              
              View camera_stream.ino
            
          
          

            
              
              Download camera_stream.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              Download Edge AI Library
            
          
        
      


      

        
Video Dithering Stream to OLED
        

          Arduino code for capturing camera frames and streaming them to an OLED display using Floyd–Steinberg dithering for high-quality monochrome rendering. This code demonstrates real-time image processing and display optimization for low-resolution OLED screens.
        


          
How the Code Works (Pseudocode):
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Initialize I²C bus for OLED display
  3. Initialize OLED display (128x64, I²C address 0x3C)
  4. Configure camera pins using camera_pins.h definitions
  5. Initialize camera with QQVGA resolution (160x120)
  6. Set pixel format to GRAYSCALE
  7. Display initialization status on OLED

LOOP:
  1. Capture camera frame using esp_camera_fb_get()
  2. Process frame through showDitheredPreview():
     a. Downsample camera frame (160x120) to OLED resolution (128x64)
     b. Average pixel values in each downsampled region
     c. Store results in gray_buffer array
     d. Normalize brightness values to full range
     e. Apply Floyd–Steinberg dithering algorithm
     f. Render dithered result to OLED display
  3. Return camera frame buffer using esp_camera_fb_return()
  4. Repeat continuously for live preview

DITHERING ALGORITHM (Floyd–Steinberg):
  For each pixel from top-left to bottom-right:
    1. Quantize current pixel (0 or 255)
    2. Calculate quantization error
    3. Distribute error to neighboring pixels:
       - Right: 7/16 of error
       - Bottom-left: 3/16 of error
       - Bottom: 5/16 of error
       - Bottom-right: 1/16 of error
        



          
Key Features:
          

            
Real-time camera frame capture at QQVGA resolution (160x120)
            
Grayscale image processing for efficient monochrome display
            
Floyd–Steinberg dithering algorithm for high-quality visual output
            
Automatic brightness normalization for optimal contrast
            
Continuous live preview stream to OLED display
            
Optimized downsampling from camera resolution to OLED resolution (128x64)
          
        


          
What You Need to Know:
          

            This code provides a foundation for displaying camera feeds on OLED displays. The Floyd–Steinberg dithering algorithm distributes quantization errors across neighboring pixels, creating smooth gradients and improved visual quality compared to simple thresholding. The code uses grayscale format for efficient processing, and the downsampling step averages multiple camera pixels into each OLED pixel to maintain image clarity.
          
          

            This implementation serves as a building block for the CircleSquareCamera AR system, where camera feeds are combined with game state overlays. See Ethical AI Use section for ChatGPT co-development transcript.
          
        


          

            
              
              View view_camera_oled.ino
            
          
          

            
              
              Download view_camera_oled.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              View ChatGPT Transcript
            
          
        
      


      

        
Group Assignment: CircleSquareCamera Augmented Reality System
        

          Complete Arduino code for the CircleSquareCamera AR system, including game board code that broadcasts player states via ESP-NOW, and camera board code that receives game packets and overlays player positions on live camera feeds.
        


        

          
Camera Board Code (CircleSquareCamera.ino)
          

            Camera board receives ESP-NOW packets from game boards, captures camera frames, processes them through dithering, and overlays game character positions (square for P1, circle for P2) on the live feed displayed on OLED.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, I²C, OLED display
  2. Initialize ESP-NOW in receive-only mode
  3. Register ESP-NOW receive callback
  4. Initialize camera (QQVGA, GRAYSCALE)
  5. Determine camera ID from MAC address (A or B)
  6. Display camera ready status

ESPNOW_RECEIVE_CALLBACK:
  1. Check if packet is correct size (NetPacket)
  2. Extract player state from packet
  3. Identify player (P1 or P2) based on sender MAC address
  4. Store player state in p1 or p2 variable
  5. Set gotP1 or gotP2 flag

LOOP:
  1. Capture camera frame (esp_camera_fb_get)
  2. Process frame through makeFrame():
     a. Downsample camera (160x120) to 64x32
     b. Normalize brightness
     c. Apply Floyd–Steinberg dithering
     d. Rotate 180° for correct orientation
  3. Draw camera feed to OLED (2x2 pixel blocks)
  4. Overlay game characters if received:
     - Draw square at p1.x, p1.y for Player 1
     - Draw circle at p2.x, p2.y for Player 2
  5. Display frame on OLED
  6. Return camera frame buffer
          



            
Key Features:
            

              
ESP-NOW receive-only mode for game state packets
              
MAC address-based player identification (P1: 0xAC, P2: 0x9C)
              
Real-time camera frame capture and dithering
              
180° rotation for correct display orientation
              
AR overlay of game characters on live camera feed
              
Automatic camera ID detection from MAC address
            
          


            

              
                
                View CircleSquareCamera.ino
              
            
            

              
                
                Download CircleSquareCamera.zip
              
            
            

              
                
                View camera_pins.h
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        


        

          
Game Board Code (TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino)
          

            Extended version of the CircleSquareShapeSongSwingalong game that broadcasts player state packets to both peer game boards and camera boards via ESP-NOW. This enables real-time AR visualization of gameplay on camera displays.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, OLED display
  2. Set WiFi mode to STA
  3. Read self MAC address
  4. Initialize ESP-NOW
  5. Identify player role (P1 or P2) from MAC address
  6. Add peers: peer game board + Camera A + Camera B
  7. Register ESP-NOW send/receive callbacks
  8. Initialize touch sensors (warm-up readings)
  9. Reset player states (left/right positioning)

LOOP:
  1. Update touch sensor readings
  2. Process player input (left, right, jump)
  3. Update player physics (movement, gravity, collision)
  4. Create NetPacket with player state and ready flag
  5. Broadcast packet to all peers:
     - Other game board (peerMac)
     - Camera A (camA_MAC)
     - Camera B (camB_MAC)
  6. Render game state to OLED:
     - Draw ground line
     - Draw own character (square if P1, circle if P2)
     - Draw other player character
     - Display "Waiting Player" if other not ready
  7. Delay for frame timing (30ms)
          



            
Key Features:
            

              
ESP-NOW one-to-many broadcast to game boards and camera boards
              
MAC address-based player identification (P1: D8:3B:DA:75:05:AC, P2: D8:3B:DA:75:E1:9C)
              
Touch-based input controls with threshold detection
              
Real-time physics simulation (movement, gravity, collision)
              
Ready state synchronization for coordinated gameplay start
              
Game state rendering with square/circle character representation
            
          


            
What You Need to Know:
            

              This code extends the original CircleSquareShapeSongSwingalong game by adding camera board peers to the ESP-NOW network. The game broadcasts player state packets (position, velocity, onGround flag) to enable AR overlay on camera displays. The camera boards use these packets to draw game characters at the correct positions relative to the camera feed, creating an augmented reality experience. Player identification is done via MAC address comparison, with P1 using square representation and P2 using circle representation. See Ethical AI Use section for ChatGPT co-development transcript.
            
          


            

              
                
                View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino
              
            
            

              
                
                Download TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.zip
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        
      
    


    

      
Reflections & Learnings
      
Key insights and learnings from working with networking and communications protocols.
    


    

      
Contributions
      
Acknowledgements and team roles for networking and communications work.
      

        

          ESP-NOW Multiplayer Game
          
Developed CircleSquareShapeSongSwingalong multiplayer game with ESP-NOW wireless communication during Week 2. See Week 2 documentation for details.
        
        

          Wi-Fi Camera Streaming System
          
Developed camera livestreaming and Edge AI face detection system during Week 10. See Week 10 documentation for details.
        
      
    


    

      
Ethical AI Use
      
Transparent documentation of AI assistance used in this week's networking and communications work.


        
AI-Assisted Camera Integration Game Update
        

          Used ChatGPT for collaborative development of the CircleSquareCamera augmented reality system, integrating ESP-NOW communication between game boards and camera boards. The AI assisted with ESP-NOW packet structure design, camera feed dithering algorithms, AR overlay implementation, and real-time synchronization of player states with camera displays. This extends the original CircleSquareShapeSongSwingalong multiplayer game from Week 2 with augmented reality visualization.
        
        

          
ChatGPT Co-Development Transcript:
          

            
              
              View ChatGPT Transcript
            
          
        
      


        
AI-Assisted Week 11 Page Creation and Assignment Documentation
        

          Cursor AI assisted with creating the complete Week 11 documentation page, including networking protocols overview, group assignment CircleSquareCamera AR documentation, individual assignment sections summarizing ESP-NOW multiplayer game and Wi-Fi camera streaming systems, network address tables, and all formatting and styling. The AI helped refine and organize content from Week 2 and Week 10 into professional, succinct networking documentation while preserving key technical details.
        
        

          The AI was used to:
        
        

          
Create week11.html from the week8.html template structure
          
Populate networking and communications resources from the MIT Academy class page
          
Summarize and integrate content from Week 2 (ESP-NOW multiplayer game) and Week 10 (Wi-Fi camera streaming)
          
Create network address tables with MAC and IP addresses for both networked systems
          
Format and style all sections for professional presentation
          
Embed images, videos, and design file download links
        
        

          
Complete AI Assistance Transcript:
          

            
              
              View HTML Transcript
            
            
              
              Download Markdown
            
          
        
      
    


      

        

          
            
          
          This work is licensed under a
          
            Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
          
        
      
    


  

Checking for lint errors and verifying the image file exists:

Refining the development note for clarity and flow:

  
  
  
  
  


  

    

      ← Back to Home
    


      

        
      
      

        
Week 11
        
Networking & Communications
      
      
    


    

      

        
        Week Highlights
      
      
Key captures from the group and individual assignments with links to their sections.


      

        
        

          
ESP-NOW Multiplayer Network
          
            
          
        
        
        

          
Wi-Fi Camera Livestream
          
            
          
        
        
        

          
CircleSquareCamera AR
          
            
          
        
      
    


    

      

        
        Table of Contents
      
      

        

          
Course Content
          

            
• Week Highlights
            
• Intro
            
• This Week's Goals
            
• Assignments
            
• Tools & Materials
          
        
        

          
Training & Documentation
          

            
• Training Documentation
            
• Recitation Notes
            
• Useful Documentation
            
• Networking Protocols
          
        
        

          
Assignments & Projects
          

            
• Group Assignment
            
• Individual Assignment
            
• Design Files
            
• Reflections & Learnings
            
• Contributions
            
• Ethical AI Use
          
        
      
    


    

      

        
Networking & Communications
        
This week focuses on connecting devices through wired and wireless networks. We'll explore communication protocols, network addressing, and how to build networked systems with local input and output devices.
        

          
Understanding wired and wireless communication protocols
          
Learning network addressing and bus architectures
          
Implementing networked nodes with input and output devices
        
      
      

        
This Week's Goals
        

          
Network Protocols
Learn about ESP-NOW, Wi-Fi, I²C, SPI, UART, and other communication protocols
          
Network Addressing
Understand MAC addresses, IP addresses, and bus addressing schemes
          
Networked Systems
Design, build, and connect wired or wireless nodes with local I/O devices
        
      
    


    

      

        
Assignments
        

          
Group Assignment
Send a message between two projects
          
Individual Assignment
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s)
        
      
      

        
Tools & Materials
        

          
ESP32-S3 Microcontroller Boards
          
Wi-Fi Access Point
          
OLED Displays (SSD1306)
          
Camera Modules
          
Touch Sensors
        
      
    


    

      
Training Documentation
      
Training materials and documentation for networking and communications protocols.


      

        
Recitation Notes
        

          Networking and communications recitation covered ESP-NOW protocol implementation, MAC address-based device identification, peer-to-peer communication setup, and network address configuration for embedded systems.
        
      


      

        
Networking Protocols Overview
        

          Networking and communications enable devices to exchange data through wired and wireless connections. Understanding different protocols helps select the right communication method for each application.
        

Wired Communication
        

          
UART/USART — Asynchronous serial communication (RS-232, RS-422, RS-485)
          
I²C/TWI — Two-wire interface with SCL/SDA, requires pull-up resistors
          
SPI — Serial Peripheral Interface with MISO/MOSI/CS/SCK lines
          
USB — Universal Serial Bus for device communication
          
Ethernet — Wired network communication
        

Wireless Communication
        

          
ESP-NOW — Low-power peer-to-peer communication for ESP32 devices
          
Wi-Fi — 802.11 wireless networking (2.4 GHz, 5.8 GHz)
          
Bluetooth/BLE — Short-range wireless communication
          
LoRa — Long-range, low-power wireless communication
          
RF — Radio frequency communication (433 MHz, 868 MHz, 915 MHz, 2.4 GHz)
        

Network Addressing
        

          Devices on networks require unique addresses for identification and communication:
        
        

          
MAC Address — Media Access Control address, unique hardware identifier (e.g., D8:3B:DA:75:05:AC)
          
IP Address — Internet Protocol address for network routing (e.g., 192.168.1.100)
          
Bus Address — Device address on shared communication buses (I²C, SPI)
        
      
    


    

      
Useful Documentation
      
Essential resources for networking and communications protocols.


        

          
Class Page
          
Networking and Communications - MIT Academy
          
Comprehensive resource covering wired protocols (UART, I²C, SPI, USB, Ethernet), wireless protocols (Wi-Fi, Bluetooth, LoRa, RF), network addressing, OSI layers, modulation techniques, channel sharing methods, error detection/correction, and Internet protocols. Includes detailed tutorials, code examples, and implementation guides for various communication protocols.
        
        

          
Key Topics
          

            
Wired Communication: UART, I²C, SPI, USB, Ethernet, CAN, LIN, MODBUS
            
Wireless Communication: ESP-NOW, Wi-Fi (802.11), Bluetooth/BLE, LoRa, RF (433/868/915 MHz, 2.4/5.8 GHz)
            
Network Protocols: IPv4/IPv6, TCP/UDP, HTTP, DNS, DHCP, NAT
            
Modulation: PCM, PPM, OOK, FSK, BPSK, QAM, OFDM, FHSS, DSSS, UWB
            
Channel Sharing: TDMA, FDMA, CSMA/CD, CSMA/CA, CDMA, MIMO
          
        
      
    


    

      
Group Assignment: Send a Message Between Two Projects
      
Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols.


        
Assignment Description
        

          Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols. The CircleSquareCamera Augmented Reality system extends the ESP-NOW multiplayer game from Week 2 by integrating camera boards that receive game state updates and overlay player positions on live camera feeds.
        
      


      

        

          
          CircleSquareCamera Augmented Reality
        


          An augmented reality system that combines the ESP-NOW multiplayer game CircleSquareShapeSongSwingalong (documented in Week 2 and Week 11 individual assignment) with camera subsystems from the final project. The system enables game boards to send player state updates via ESP-NOW to camera boards, which overlay game characters (square and circle) on live camera feeds displayed on OLED screens. Camera code development spans Week 7, Week 8, Final Project, and Week 12 (placeholder link).
        


          

            
            
          
          

            

              Development Note: Initial implementation attempted to send camera data from camera boards to game boards. After two hours of troubleshooting communication issues, the architecture was reversed: the final system sends game state packets from game boards to camera boards via ESP-NOW. This approach proved more efficient for real-time AR overlay visualization, as game state is lightweight compared to camera frame data.
            
          
          

            
              
            
            
          
        

Design & Build
        

          The system consists of two networked subsystems: game boards (XIAO ESP32-S3 with touch sensors and OLED displays) and camera boards (XIAO ESP32-S3 Sense with integrated cameras and OLED displays). Game boards run the CircleSquareShapeSongSwingalong multiplayer game, processing touch input, updating player physics, and broadcasting player state packets via ESP-NOW to both the other game board and all camera boards. Camera boards receive these packets, capture camera frames, process them through Floyd–Steinberg dithering for OLED display, and overlay game character positions (square for Player 1, circle for Player 2) on the live feed.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification for game boards and camera boards

          Communication Pattern: One-to-many broadcast from game boards to peer game board and all camera boards
        

Input & Output Devices
        

          
Game Boards:
            

              
Input: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering
            
          
          
Camera Boards:
            

              
Input: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, ESP-NOW packets for game state
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for displaying dithered camera feed with AR overlay
            
          
        

Connections
        

          Game Boards: Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. ESP-NOW communication is handled through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.


          Camera Boards: The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. ESP-NOW reception uses the ESP32-S3's built-in Wi-Fi radio in receive-only mode.
        


        

          
Network Address Tables
          

            Complete network addressing information for all devices in the CircleSquareCamera AR system, including MAC addresses (hardware identifiers) for ESP-NOW communication.
          

ESP-NOW Network (CircleSquareCamera AR System)
          

            

              

                
Device
                
Component
                
MAC Address
                
Network Type
                
Input Device
                
Output Device
              
            
            

              

                
P1
                
XIAO ESP32-S3
                
D8:3B:DA:75:05:AC
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
P2
                
XIAO ESP32-S3
                
D8:3B:DA:75:E1:9C
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
Camera A
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:E2:C0
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
              

                
Camera B
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:D6:38
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
            
          


            

              Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. Game boards broadcast player state packets to both the peer game board and all camera boards. Camera boards operate in receive-only mode, processing incoming ESP-NOW packets to identify player positions and overlay them on camera feeds.
            
          
        


          
Download Design Files
          

            Complete Arduino code for game boards and camera boards, including ESP-NOW communication, camera feed processing, and AR overlay implementation. See Design Files section for detailed documentation and download links.
          
        
      
    


    

      
Individual Assignment: Networking & Communications
      
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s). This assignment demonstrates two networked systems: an ESP-NOW multiplayer game and a Wi-Fi camera streaming system with edge AI processing.


        
Related Documentation
        

          
Week 2: Embedded Programming & Nintendo Training — ESP-NOW multiplayer game development
          
Week 2: Design Files — CircleSquareShapeSongSwingalong Arduino code
          
Week 10: Camera & Edge AI — Wi-Fi camera streaming system
          
Week 10: Design Files - Camera — Camera stream code and system implementation
        
      


      

        
Networked System 1: CircleSquareShapeSongSwingalong (ESP-NOW)
        

          A collaborative multiplayer game featuring geometric shape interactions, synchronized gameplay between two ESP32-S3 devices using ESP-NOW wireless communication. The system uses MAC address-based player identification and real-time state synchronization for coordinated gameplay. See Week 2 documentation and design files for complete details.
        

Design & Build
        

          The system consists of two XIAO ESP32-S3 development boards, each with an SSD1306 OLED display (128x64) and capacitive touch sensors. The boards communicate wirelessly using ESP-NOW protocol, enabling low-latency peer-to-peer communication without requiring a Wi-Fi access point.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification
        

Input & Output Devices
        

          
Input Devices: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
          
Output Devices: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering and player state visualization
        


          

            

              
            
            
Shape Song Swing Along educational content integration and gameplay
          
          

            

              
            
            
Circle and square shape interaction gameplay mechanics
          
        

Connections
        

          Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. The ESP-NOW communication is handled entirely through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.
        


          
Download Design Files
          

            
TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino — Complete Arduino sketch for ESP-NOW multiplayer game
            
get_mac_address.ino — Utility to read MAC addresses from ESP32-S3 devices
          
        
      


      

        
Networked System 2: Camera Livestream & Edge AI (Wi-Fi)
        

          A Wi-Fi-enabled camera system using ESP32-S3's built-in camera and HTTP server capabilities to stream JPEG frames over Wi-Fi using MJPEG (Motion JPEG) protocol. The system also includes edge AI face detection using a FOMO (Faster Objects, More Objects) model from Edge Impulse for real-time person tracking. See Week 10 documentation and design files for complete details.
        

Design & Build
        

          The system uses a XIAO ESP32-S3 Sense development board with an integrated camera module. The camera captures frames at QQVGA (160x120) or QVGA resolution, processes them through an Edge AI inference pipeline, and streams the results over Wi-Fi via HTTP multipart response. The system can also display camera output and inference results on an OLED display connected via I²C.
        

Network Type & Protocol
        

          Network Type: Wireless local area network (Wi-Fi)

          Protocol: HTTP over Wi-Fi (802.11), MJPEG streaming

          Addressing Scheme: IP address (DHCP-assigned) and MAC address (hardware identifier)
        

Input & Output Devices
        

          
Input Devices: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, button for capture trigger
          
Output Devices: OLED display (SSD1306, 128x64, I²C address 0x3C) for displaying camera frames and inference results, Wi-Fi HTTP server for remote streaming
        


          

            
            
          
        
        

          

            
            
          
          

            
            
          
          

            
            
          
        

Connections
        

          The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. Wi-Fi connection is established through the ESP32-S3's built-in Wi-Fi radio.
        


          
Download Design Files
          

            
camera_stream.zip — Complete camera stream project (includes .ino and .h files)
            
camera_stream.ino — Main Arduino sketch for camera livestreaming
            
camera_pins.h — GPIO pin definitions for XIAO ESP32-S3 camera module
            
ei-face-detection--fomo-arduino-1.0.90.zip — Edge Impulse Arduino library for FOMO-based face detection
          
        
      


      

        
Network Address Tables
        

          Complete network addressing information for all connected devices, including MAC addresses (hardware identifiers) and IP addresses (network identifiers) for Wi-Fi-connected devices.
        

ESP-NOW Network (CircleSquareShapeSongSwingalong)
        

          

            

              
Device
              
Component
              
MAC Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Player 1
              
XIAO ESP32-S3
              
D8:3B:DA:75:05:AC
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
            

              
Player 2
              
XIAO ESP32-S3
              
D8:3B:DA:75:E1:9C
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
          
        

Wi-Fi Network (Camera Livestream System)
        

          

            

              
Device
              
Component
              
MAC Address
              
IP Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Camera Node
              
XIAO ESP32-S3 Sense
              
[Device MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
Camera module, Button
              
OLED (SSD1306, 0x3C), HTTP Server
            
            

              
Client Device
              
Web Browser
              
[Client MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
N/A
              
Display (receives MJPEG stream)
            
          
        


          

            Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. IP addresses are assigned dynamically via DHCP when connecting to a Wi-Fi network. For ESP-NOW communication, only MAC addresses are used (no IP addresses required). The camera system's IP address is printed to the Serial Monitor upon Wi-Fi connection establishment.
          
        
      
    


    

      
Design Files
      
Complete design files for networked systems including ESP-NOW multiplayer game and Wi-Fi camera streaming implementations with Arduino firmware and configuration files.


      

        
ESP-NOW Multiplayer Game (CircleSquareShapeSongSwingalong)
        

          Arduino code for ESP-NOW wireless multiplayer game with MAC address-based player identification and synchronized game state management.
        
        

          
Key Features:
          

            
ESP-NOW wireless peer-to-peer communication
            
MAC address-based player identification (D8:3B:DA:75:05:AC and D8:3B:DA:75:E1:9C)
            
Real-time game state synchronization
            
Ready state coordination between players
            
Touch-based input controls (left, right, jump)
            
OLED display output for game rendering
          
        


          

            
              
              View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino
            
          
          

            
              
              Download .ino
            
          
          

            
              
              Download get_mac_address.ino
            
          
        
      


      

        
Wi-Fi Camera Livestream & Edge AI
        

          Arduino code for ESP32-S3 camera livestreaming over Wi-Fi using MJPEG protocol and Edge AI face detection using FOMO models from Edge Impulse.
        


          
Camera Livestream Pseudocode:
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Configure camera pins (from camera_pins.h)
  3. Create camera_config_t structure
  4. Initialize camera with esp_camera_init()
  5. Connect to Wi-Fi network
  6. Start HTTP server with stream handler

STREAM_HANDLER:
  1. Set HTTP response type to "multipart/x-mixed-replace"
  2. Enter loop: capture frame, send via HTTP, repeat

LOOP:
  - Minimal delay to allow other tasks
        



          
Key Features:
          

            
Wi-Fi HTTP server for MJPEG streaming
            
Edge AI face detection using FOMO models
            
Real-time camera frame capture and processing
            
OLED display output for local visualization
            
IP address assignment via DHCP
            
MAC address for hardware identification
          
        


          

            
              
              View camera_stream.ino
            
          
          

            
              
              Download camera_stream.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              Download Edge AI Library
            
          
        
      


      

        
Video Dithering Stream to OLED
        

          Arduino code for capturing camera frames and streaming them to an OLED display using Floyd–Steinberg dithering for high-quality monochrome rendering. This code demonstrates real-time image processing and display optimization for low-resolution OLED screens.
        


          
How the Code Works (Pseudocode):
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Initialize I²C bus for OLED display
  3. Initialize OLED display (128x64, I²C address 0x3C)
  4. Configure camera pins using camera_pins.h definitions
  5. Initialize camera with QQVGA resolution (160x120)
  6. Set pixel format to GRAYSCALE
  7. Display initialization status on OLED

LOOP:
  1. Capture camera frame using esp_camera_fb_get()
  2. Process frame through showDitheredPreview():
     a. Downsample camera frame (160x120) to OLED resolution (128x64)
     b. Average pixel values in each downsampled region
     c. Store results in gray_buffer array
     d. Normalize brightness values to full range
     e. Apply Floyd–Steinberg dithering algorithm
     f. Render dithered result to OLED display
  3. Return camera frame buffer using esp_camera_fb_return()
  4. Repeat continuously for live preview

DITHERING ALGORITHM (Floyd–Steinberg):
  For each pixel from top-left to bottom-right:
    1. Quantize current pixel (0 or 255)
    2. Calculate quantization error
    3. Distribute error to neighboring pixels:
       - Right: 7/16 of error
       - Bottom-left: 3/16 of error
       - Bottom: 5/16 of error
       - Bottom-right: 1/16 of error
        



          
Key Features:
          

            
Real-time camera frame capture at QQVGA resolution (160x120)
            
Grayscale image processing for efficient monochrome display
            
Floyd–Steinberg dithering algorithm for high-quality visual output
            
Automatic brightness normalization for optimal contrast
            
Continuous live preview stream to OLED display
            
Optimized downsampling from camera resolution to OLED resolution (128x64)
          
        


          
What You Need to Know:
          

            This code provides a foundation for displaying camera feeds on OLED displays. The Floyd–Steinberg dithering algorithm distributes quantization errors across neighboring pixels, creating smooth gradients and improved visual quality compared to simple thresholding. The code uses grayscale format for efficient processing, and the downsampling step averages multiple camera pixels into each OLED pixel to maintain image clarity.
          
          

            This implementation serves as a building block for the CircleSquareCamera AR system, where camera feeds are combined with game state overlays. See Ethical AI Use section for ChatGPT co-development transcript.
          
        


          

            
              
              View view_camera_oled.ino
            
          
          

            
              
              Download view_camera_oled.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              View ChatGPT Transcript
            
          
        
      


      

        
Group Assignment: CircleSquareCamera Augmented Reality System
        

          Complete Arduino code for the CircleSquareCamera AR system, including game board code that broadcasts player states via ESP-NOW, and camera board code that receives game packets and overlays player positions on live camera feeds.
        


        

          
Camera Board Code (CircleSquareCamera.ino)
          

            Camera board receives ESP-NOW packets from game boards, captures camera frames, processes them through dithering, and overlays game character positions (square for P1, circle for P2) on the live feed displayed on OLED.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, I²C, OLED display
  2. Initialize ESP-NOW in receive-only mode
  3. Register ESP-NOW receive callback
  4. Initialize camera (QQVGA, GRAYSCALE)
  5. Determine camera ID from MAC address (A or B)
  6. Display camera ready status

ESPNOW_RECEIVE_CALLBACK:
  1. Check if packet is correct size (NetPacket)
  2. Extract player state from packet
  3. Identify player (P1 or P2) based on sender MAC address
  4. Store player state in p1 or p2 variable
  5. Set gotP1 or gotP2 flag

LOOP:
  1. Capture camera frame (esp_camera_fb_get)
  2. Process frame through makeFrame():
     a. Downsample camera (160x120) to 64x32
     b. Normalize brightness
     c. Apply Floyd–Steinberg dithering
     d. Rotate 180° for correct orientation
  3. Draw camera feed to OLED (2x2 pixel blocks)
  4. Overlay game characters if received:
     - Draw square at p1.x, p1.y for Player 1
     - Draw circle at p2.x, p2.y for Player 2
  5. Display frame on OLED
  6. Return camera frame buffer
          



            
Key Features:
            

              
ESP-NOW receive-only mode for game state packets
              
MAC address-based player identification (P1: 0xAC, P2: 0x9C)
              
Real-time camera frame capture and dithering
              
180° rotation for correct display orientation
              
AR overlay of game characters on live camera feed
              
Automatic camera ID detection from MAC address
            
          


            

              
                
                View CircleSquareCamera.ino
              
            
            

              
                
                Download CircleSquareCamera.zip
              
            
            

              
                
                View camera_pins.h
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        


        

          
Game Board Code (TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino)
          

            Extended version of the CircleSquareShapeSongSwingalong game that broadcasts player state packets to both peer game boards and camera boards via ESP-NOW. This enables real-time AR visualization of gameplay on camera displays.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, OLED display
  2. Set WiFi mode to STA
  3. Read self MAC address
  4. Initialize ESP-NOW
  5. Identify player role (P1 or P2) from MAC address
  6. Add peers: peer game board + Camera A + Camera B
  7. Register ESP-NOW send/receive callbacks
  8. Initialize touch sensors (warm-up readings)
  9. Reset player states (left/right positioning)

LOOP:
  1. Update touch sensor readings
  2. Process player input (left, right, jump)
  3. Update player physics (movement, gravity, collision)
  4. Create NetPacket with player state and ready flag
  5. Broadcast packet to all peers:
     - Other game board (peerMac)
     - Camera A (camA_MAC)
     - Camera B (camB_MAC)
  6. Render game state to OLED:
     - Draw ground line
     - Draw own character (square if P1, circle if P2)
     - Draw other player character
     - Display "Waiting Player" if other not ready
  7. Delay for frame timing (30ms)
          



            
Key Features:
            

              
ESP-NOW one-to-many broadcast to game boards and camera boards
              
MAC address-based player identification (P1: D8:3B:DA:75:05:AC, P2: D8:3B:DA:75:E1:9C)
              
Touch-based input controls with threshold detection
              
Real-time physics simulation (movement, gravity, collision)
              
Ready state synchronization for coordinated gameplay start
              
Game state rendering with square/circle character representation
            
          


            
What You Need to Know:
            

              This code extends the original CircleSquareShapeSongSwingalong game by adding camera board peers to the ESP-NOW network. The game broadcasts player state packets (position, velocity, onGround flag) to enable AR overlay on camera displays. The camera boards use these packets to draw game characters at the correct positions relative to the camera feed, creating an augmented reality experience. Player identification is done via MAC address comparison, with P1 using square representation and P2 using circle representation. See Ethical AI Use section for ChatGPT co-development transcript.
            
          


            

              
                
                View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino
              
            
            

              
                
                Download TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.zip
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        
      
    


    

      
Reflections & Learnings
      
Key insights and learnings from working with networking and communications protocols.
    


    

      
Contributions
      
Acknowledgements and team roles for networking and communications work.
      

        

          ESP-NOW Multiplayer Game
          
Developed CircleSquareShapeSongSwingalong multiplayer game with ESP-NOW wireless communication during Week 2. See Week 2 documentation for details.
        
        

          Wi-Fi Camera Streaming System
          
Developed camera livestreaming and Edge AI face detection system during Week 10. See Week 10 documentation for details.
        
      
    


    

      
Ethical AI Use
      
Transparent documentation of AI assistance used in this week's networking and communications work.


        
AI-Assisted Camera Integration Game Update
        

          Used ChatGPT for collaborative development of the CircleSquareCamera augmented reality system, integrating ESP-NOW communication between game boards and camera boards. The AI assisted with ESP-NOW packet structure design, camera feed dithering algorithms, AR overlay implementation, and real-time synchronization of player states with camera displays. This extends the original CircleSquareShapeSongSwingalong multiplayer game from Week 2 with augmented reality visualization.
        
        

          
ChatGPT Co-Development Transcript:
          

            
              
              View ChatGPT Transcript
            
          
        
      


        
AI-Assisted Week 11 Page Creation and Assignment Documentation
        

          Cursor AI assisted with creating the complete Week 11 documentation page, including networking protocols overview, group assignment CircleSquareCamera AR documentation, individual assignment sections summarizing ESP-NOW multiplayer game and Wi-Fi camera streaming systems, network address tables, and all formatting and styling. The AI helped refine and organize content from Week 2 and Week 10 into professional, succinct networking documentation while preserving key technical details.
        
        

          The AI was used to:
        
        

          
Create week11.html from the week8.html template structure
          
Populate networking and communications resources from the MIT Academy class page
          
Summarize and integrate content from Week 2 (ESP-NOW multiplayer game) and Week 10 (Wi-Fi camera streaming)
          
Create network address tables with MAC and IP addresses for both networked systems
          
Format and style all sections for professional presentation
          
Embed images, videos, and design file download links
        
        

          
Complete AI Assistance Transcript:
          

            
              
              View HTML Transcript
            
            
              
              Download Markdown
            
          
        
      
    


      

        

          
            
          
          This work is licensed under a
          
            Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
          
        
      
    


  

Updated the CircleSquareCamera AR section:

1. Refined video caption — Added a development note explaining the architectural change from sending camera data to game boards, to sending game state to camera boards, after two hours of troubleshooting.

2. Added Magic School Bus quote image — Included the image (if_at_first_you_dont_succeed_find_out_why.png) below the video with: - Clickable link to the Reddit source - Caption acknowledging Quentin for the quote on the HTMAA XIAO board - Styled to match the page design

The development note is in a highlighted box that draws attention while staying concise. The quote image is properly linked and formatted.

All changes are complete and ready. The page now documents the development process and acknowledges Quentin's contribution.

Locating the "Download Design Files" section under Group Assignment to add the links:

Checking section anchors in the Design Files section to create the correct links:

Adding ID attributes to the relevant sections for linking. Updating the "Download Design Files" section under Group Assignment with specific links:

Adding ID attributes to the relevant sections, then updating the "Download Design Files" section under Group Assignment with specific links:

  
  
  
  
  


  

    

      ← Back to Home
    


      

        
      
      

        
Week 11
        
Networking & Communications
      
      
    


    

      

        
        Week Highlights
      
      
Key captures from the group and individual assignments with links to their sections.


      

        
        

          
ESP-NOW Multiplayer Network
          
            
          
        
        
        

          
Wi-Fi Camera Livestream
          
            
          
        
        
        

          
CircleSquareCamera AR
          
            
          
        
      
    


    

      

        
        Table of Contents
      
      

        

          
Course Content
          

            
• Week Highlights
            
• Intro
            
• This Week's Goals
            
• Assignments
            
• Tools & Materials
          
        
        

          
Training & Documentation
          

            
• Training Documentation
            
• Recitation Notes
            
• Useful Documentation
            
• Networking Protocols
          
        
        

          
Assignments & Projects
          

            
• Group Assignment
            
• Individual Assignment
            
• Design Files
            
• Reflections & Learnings
            
• Contributions
            
• Ethical AI Use
          
        
      
    


    

      

        
Networking & Communications
        
This week focuses on connecting devices through wired and wireless networks. We'll explore communication protocols, network addressing, and how to build networked systems with local input and output devices.
        

          
Understanding wired and wireless communication protocols
          
Learning network addressing and bus architectures
          
Implementing networked nodes with input and output devices
        
      
      

        
This Week's Goals
        

          
Network Protocols
Learn about ESP-NOW, Wi-Fi, I²C, SPI, UART, and other communication protocols
          
Network Addressing
Understand MAC addresses, IP addresses, and bus addressing schemes
          
Networked Systems
Design, build, and connect wired or wireless nodes with local I/O devices
        
      
    


    

      

        
Assignments
        

          
Group Assignment
Send a message between two projects
          
Individual Assignment
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s)
        
      
      

        
Tools & Materials
        

          
ESP32-S3 Microcontroller Boards
          
Wi-Fi Access Point
          
OLED Displays (SSD1306)
          
Camera Modules
          
Touch Sensors
        
      
    


    

      
Training Documentation
      
Training materials and documentation for networking and communications protocols.


      

        
Recitation Notes
        

          Networking and communications recitation covered ESP-NOW protocol implementation, MAC address-based device identification, peer-to-peer communication setup, and network address configuration for embedded systems.
        
      


      

        
Networking Protocols Overview
        

          Networking and communications enable devices to exchange data through wired and wireless connections. Understanding different protocols helps select the right communication method for each application.
        

Wired Communication
        

          
UART/USART — Asynchronous serial communication (RS-232, RS-422, RS-485)
          
I²C/TWI — Two-wire interface with SCL/SDA, requires pull-up resistors
          
SPI — Serial Peripheral Interface with MISO/MOSI/CS/SCK lines
          
USB — Universal Serial Bus for device communication
          
Ethernet — Wired network communication
        

Wireless Communication
        

          
ESP-NOW — Low-power peer-to-peer communication for ESP32 devices
          
Wi-Fi — 802.11 wireless networking (2.4 GHz, 5.8 GHz)
          
Bluetooth/BLE — Short-range wireless communication
          
LoRa — Long-range, low-power wireless communication
          
RF — Radio frequency communication (433 MHz, 868 MHz, 915 MHz, 2.4 GHz)
        

Network Addressing
        

          Devices on networks require unique addresses for identification and communication:
        
        

          
MAC Address — Media Access Control address, unique hardware identifier (e.g., D8:3B:DA:75:05:AC)
          
IP Address — Internet Protocol address for network routing (e.g., 192.168.1.100)
          
Bus Address — Device address on shared communication buses (I²C, SPI)
        
      
    


    

      
Useful Documentation
      
Essential resources for networking and communications protocols.


        

          
Class Page
          
Networking and Communications - MIT Academy
          
Comprehensive resource covering wired protocols (UART, I²C, SPI, USB, Ethernet), wireless protocols (Wi-Fi, Bluetooth, LoRa, RF), network addressing, OSI layers, modulation techniques, channel sharing methods, error detection/correction, and Internet protocols. Includes detailed tutorials, code examples, and implementation guides for various communication protocols.
        
        

          
Key Topics
          

            
Wired Communication: UART, I²C, SPI, USB, Ethernet, CAN, LIN, MODBUS
            
Wireless Communication: ESP-NOW, Wi-Fi (802.11), Bluetooth/BLE, LoRa, RF (433/868/915 MHz, 2.4/5.8 GHz)
            
Network Protocols: IPv4/IPv6, TCP/UDP, HTTP, DNS, DHCP, NAT
            
Modulation: PCM, PPM, OOK, FSK, BPSK, QAM, OFDM, FHSS, DSSS, UWB
            
Channel Sharing: TDMA, FDMA, CSMA/CD, CSMA/CA, CDMA, MIMO
          
        
      
    


    

      
Group Assignment: Send a Message Between Two Projects
      
Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols.


        
Assignment Description
        

          Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols. The CircleSquareCamera Augmented Reality system extends the ESP-NOW multiplayer game from Week 2 by integrating camera boards that receive game state updates and overlay player positions on live camera feeds.
        
      


      

        

          
          CircleSquareCamera Augmented Reality
        


          An augmented reality system that combines the ESP-NOW multiplayer game CircleSquareShapeSongSwingalong (documented in Week 2 and Week 11 individual assignment) with camera subsystems from the final project. The system enables game boards to send player state updates via ESP-NOW to camera boards, which overlay game characters (square and circle) on live camera feeds displayed on OLED screens. Camera code development spans Week 7, Week 8, Final Project, and Week 12 (placeholder link).
        


          

            
            
          
          

            

              Development Note: Initial implementation attempted to send camera data from camera boards to game boards. After two hours of troubleshooting communication issues, the architecture was reversed: the final system sends game state packets from game boards to camera boards via ESP-NOW. This approach proved more efficient for real-time AR overlay visualization, as game state is lightweight compared to camera frame data.
            
          
          

            
              
            
            
          
        

Design & Build
        

          The system consists of two networked subsystems: game boards (XIAO ESP32-S3 with touch sensors and OLED displays) and camera boards (XIAO ESP32-S3 Sense with integrated cameras and OLED displays). Game boards run the CircleSquareShapeSongSwingalong multiplayer game, processing touch input, updating player physics, and broadcasting player state packets via ESP-NOW to both the other game board and all camera boards. Camera boards receive these packets, capture camera frames, process them through Floyd–Steinberg dithering for OLED display, and overlay game character positions (square for Player 1, circle for Player 2) on the live feed.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification for game boards and camera boards

          Communication Pattern: One-to-many broadcast from game boards to peer game board and all camera boards
        

Input & Output Devices
        

          
Game Boards:
            

              
Input: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering
            
          
          
Camera Boards:
            

              
Input: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, ESP-NOW packets for game state
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for displaying dithered camera feed with AR overlay
            
          
        

Connections
        

          Game Boards: Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. ESP-NOW communication is handled through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.


          Camera Boards: The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. ESP-NOW reception uses the ESP32-S3's built-in Wi-Fi radio in receive-only mode.
        


        

          
Network Address Tables
          

            Complete network addressing information for all devices in the CircleSquareCamera AR system, including MAC addresses (hardware identifiers) for ESP-NOW communication.
          

ESP-NOW Network (CircleSquareCamera AR System)
          

            

              

                
Device
                
Component
                
MAC Address
                
Network Type
                
Input Device
                
Output Device
              
            
            

              

                
P1
                
XIAO ESP32-S3
                
D8:3B:DA:75:05:AC
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
P2
                
XIAO ESP32-S3
                
D8:3B:DA:75:E1:9C
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
Camera A
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:E2:C0
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
              

                
Camera B
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:D6:38
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
            
          


            

              Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. Game boards broadcast player state packets to both the peer game board and all camera boards. Camera boards operate in receive-only mode, processing incoming ESP-NOW packets to identify player positions and overlay them on camera feeds.
            
          
        


          
Download Design Files
          

            Complete Arduino code for game boards and camera boards, including ESP-NOW communication, camera feed processing, and AR overlay implementation. See Design Files section for detailed documentation and download links.
          
        
      
    


    

      
Individual Assignment: Networking & Communications
      
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s). This assignment demonstrates two networked systems: an ESP-NOW multiplayer game and a Wi-Fi camera streaming system with edge AI processing.


        
Related Documentation
        

          
Week 2: Embedded Programming & Nintendo Training — ESP-NOW multiplayer game development
          
Week 2: Design Files — CircleSquareShapeSongSwingalong Arduino code
          
Week 10: Camera & Edge AI — Wi-Fi camera streaming system
          
Week 10: Design Files - Camera — Camera stream code and system implementation
        
      


      

        
Networked System 1: CircleSquareShapeSongSwingalong (ESP-NOW)
        

          A collaborative multiplayer game featuring geometric shape interactions, synchronized gameplay between two ESP32-S3 devices using ESP-NOW wireless communication. The system uses MAC address-based player identification and real-time state synchronization for coordinated gameplay. See Week 2 documentation and design files for complete details.
        

Design & Build
        

          The system consists of two XIAO ESP32-S3 development boards, each with an SSD1306 OLED display (128x64) and capacitive touch sensors. The boards communicate wirelessly using ESP-NOW protocol, enabling low-latency peer-to-peer communication without requiring a Wi-Fi access point.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification
        

Input & Output Devices
        

          
Input Devices: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
          
Output Devices: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering and player state visualization
        


          

            

              
            
            
Shape Song Swing Along educational content integration and gameplay
          
          

            

              
            
            
Circle and square shape interaction gameplay mechanics
          
        

Connections
        

          Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. The ESP-NOW communication is handled entirely through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.
        


          
Download Design Files
          

            
TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino — Complete Arduino sketch for ESP-NOW multiplayer game
            
get_mac_address.ino — Utility to read MAC addresses from ESP32-S3 devices
          
        
      


      

        
Networked System 2: Camera Livestream & Edge AI (Wi-Fi)
        

          A Wi-Fi-enabled camera system using ESP32-S3's built-in camera and HTTP server capabilities to stream JPEG frames over Wi-Fi using MJPEG (Motion JPEG) protocol. The system also includes edge AI face detection using a FOMO (Faster Objects, More Objects) model from Edge Impulse for real-time person tracking. See Week 10 documentation and design files for complete details.
        

Design & Build
        

          The system uses a XIAO ESP32-S3 Sense development board with an integrated camera module. The camera captures frames at QQVGA (160x120) or QVGA resolution, processes them through an Edge AI inference pipeline, and streams the results over Wi-Fi via HTTP multipart response. The system can also display camera output and inference results on an OLED display connected via I²C.
        

Network Type & Protocol
        

          Network Type: Wireless local area network (Wi-Fi)

          Protocol: HTTP over Wi-Fi (802.11), MJPEG streaming

          Addressing Scheme: IP address (DHCP-assigned) and MAC address (hardware identifier)
        

Input & Output Devices
        

          
Input Devices: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, button for capture trigger
          
Output Devices: OLED display (SSD1306, 128x64, I²C address 0x3C) for displaying camera frames and inference results, Wi-Fi HTTP server for remote streaming
        


          

            
            
          
        
        

          

            
            
          
          

            
            
          
          

            
            
          
        

Connections
        

          The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. Wi-Fi connection is established through the ESP32-S3's built-in Wi-Fi radio.
        


          
Download Design Files
          

            
camera_stream.zip — Complete camera stream project (includes .ino and .h files)
            
camera_stream.ino — Main Arduino sketch for camera livestreaming
            
camera_pins.h — GPIO pin definitions for XIAO ESP32-S3 camera module
            
ei-face-detection--fomo-arduino-1.0.90.zip — Edge Impulse Arduino library for FOMO-based face detection
          
        
      


      

        
Network Address Tables
        

          Complete network addressing information for all connected devices, including MAC addresses (hardware identifiers) and IP addresses (network identifiers) for Wi-Fi-connected devices.
        

ESP-NOW Network (CircleSquareShapeSongSwingalong)
        

          

            

              
Device
              
Component
              
MAC Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Player 1
              
XIAO ESP32-S3
              
D8:3B:DA:75:05:AC
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
            

              
Player 2
              
XIAO ESP32-S3
              
D8:3B:DA:75:E1:9C
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
          
        

Wi-Fi Network (Camera Livestream System)
        

          

            

              
Device
              
Component
              
MAC Address
              
IP Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Camera Node
              
XIAO ESP32-S3 Sense
              
[Device MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
Camera module, Button
              
OLED (SSD1306, 0x3C), HTTP Server
            
            

              
Client Device
              
Web Browser
              
[Client MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
N/A
              
Display (receives MJPEG stream)
            
          
        


          

            Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. IP addresses are assigned dynamically via DHCP when connecting to a Wi-Fi network. For ESP-NOW communication, only MAC addresses are used (no IP addresses required). The camera system's IP address is printed to the Serial Monitor upon Wi-Fi connection establishment.
          
        
      
    


    

      
Design Files
      
Complete design files for networked systems including ESP-NOW multiplayer game and Wi-Fi camera streaming implementations with Arduino firmware and configuration files.


      

        
ESP-NOW Multiplayer Game (CircleSquareShapeSongSwingalong)
        

          Arduino code for ESP-NOW wireless multiplayer game with MAC address-based player identification and synchronized game state management.
        
        

          
Key Features:
          

            
ESP-NOW wireless peer-to-peer communication
            
MAC address-based player identification (D8:3B:DA:75:05:AC and D8:3B:DA:75:E1:9C)
            
Real-time game state synchronization
            
Ready state coordination between players
            
Touch-based input controls (left, right, jump)
            
OLED display output for game rendering
          
        


          

            
              
              View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino
            
          
          

            
              
              Download .ino
            
          
          

            
              
              Download get_mac_address.ino
            
          
        
      


      

        
Wi-Fi Camera Livestream & Edge AI
        

          Arduino code for ESP32-S3 camera livestreaming over Wi-Fi using MJPEG protocol and Edge AI face detection using FOMO models from Edge Impulse.
        


          
Camera Livestream Pseudocode:
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Configure camera pins (from camera_pins.h)
  3. Create camera_config_t structure
  4. Initialize camera with esp_camera_init()
  5. Connect to Wi-Fi network
  6. Start HTTP server with stream handler

STREAM_HANDLER:
  1. Set HTTP response type to "multipart/x-mixed-replace"
  2. Enter loop: capture frame, send via HTTP, repeat

LOOP:
  - Minimal delay to allow other tasks
        



          
Key Features:
          

            
Wi-Fi HTTP server for MJPEG streaming
            
Edge AI face detection using FOMO models
            
Real-time camera frame capture and processing
            
OLED display output for local visualization
            
IP address assignment via DHCP
            
MAC address for hardware identification
          
        


          

            
              
              View camera_stream.ino
            
          
          

            
              
              Download camera_stream.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              Download Edge AI Library
            
          
        
      


      

        
Video Dithering Stream to OLED
        

          Arduino code for capturing camera frames and streaming them to an OLED display using Floyd–Steinberg dithering for high-quality monochrome rendering. This code demonstrates real-time image processing and display optimization for low-resolution OLED screens.
        


          
How the Code Works (Pseudocode):
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Initialize I²C bus for OLED display
  3. Initialize OLED display (128x64, I²C address 0x3C)
  4. Configure camera pins using camera_pins.h definitions
  5. Initialize camera with QQVGA resolution (160x120)
  6. Set pixel format to GRAYSCALE
  7. Display initialization status on OLED

LOOP:
  1. Capture camera frame using esp_camera_fb_get()
  2. Process frame through showDitheredPreview():
     a. Downsample camera frame (160x120) to OLED resolution (128x64)
     b. Average pixel values in each downsampled region
     c. Store results in gray_buffer array
     d. Normalize brightness values to full range
     e. Apply Floyd–Steinberg dithering algorithm
     f. Render dithered result to OLED display
  3. Return camera frame buffer using esp_camera_fb_return()
  4. Repeat continuously for live preview

DITHERING ALGORITHM (Floyd–Steinberg):
  For each pixel from top-left to bottom-right:
    1. Quantize current pixel (0 or 255)
    2. Calculate quantization error
    3. Distribute error to neighboring pixels:
       - Right: 7/16 of error
       - Bottom-left: 3/16 of error
       - Bottom: 5/16 of error
       - Bottom-right: 1/16 of error
        



          
Key Features:
          

            
Real-time camera frame capture at QQVGA resolution (160x120)
            
Grayscale image processing for efficient monochrome display
            
Floyd–Steinberg dithering algorithm for high-quality visual output
            
Automatic brightness normalization for optimal contrast
            
Continuous live preview stream to OLED display
            
Optimized downsampling from camera resolution to OLED resolution (128x64)
          
        


          
What You Need to Know:
          

            This code provides a foundation for displaying camera feeds on OLED displays. The Floyd–Steinberg dithering algorithm distributes quantization errors across neighboring pixels, creating smooth gradients and improved visual quality compared to simple thresholding. The code uses grayscale format for efficient processing, and the downsampling step averages multiple camera pixels into each OLED pixel to maintain image clarity.
          
          

            This implementation serves as a building block for the CircleSquareCamera AR system, where camera feeds are combined with game state overlays. See Ethical AI Use section for ChatGPT co-development transcript.
          
        


          

            
              
              View view_camera_oled.ino
            
          
          

            
              
              Download view_camera_oled.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              View ChatGPT Transcript
            
          
        
      


      

        
Group Assignment: CircleSquareCamera Augmented Reality System
        

          Complete Arduino code for the CircleSquareCamera AR system, including game board code that broadcasts player states via ESP-NOW, and camera board code that receives game packets and overlays player positions on live camera feeds.
        


        

          
Camera Board Code (CircleSquareCamera.ino)
          

            Camera board receives ESP-NOW packets from game boards, captures camera frames, processes them through dithering, and overlays game character positions (square for P1, circle for P2) on the live feed displayed on OLED.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, I²C, OLED display
  2. Initialize ESP-NOW in receive-only mode
  3. Register ESP-NOW receive callback
  4. Initialize camera (QQVGA, GRAYSCALE)
  5. Determine camera ID from MAC address (A or B)
  6. Display camera ready status

ESPNOW_RECEIVE_CALLBACK:
  1. Check if packet is correct size (NetPacket)
  2. Extract player state from packet
  3. Identify player (P1 or P2) based on sender MAC address
  4. Store player state in p1 or p2 variable
  5. Set gotP1 or gotP2 flag

LOOP:
  1. Capture camera frame (esp_camera_fb_get)
  2. Process frame through makeFrame():
     a. Downsample camera (160x120) to 64x32
     b. Normalize brightness
     c. Apply Floyd–Steinberg dithering
     d. Rotate 180° for correct orientation
  3. Draw camera feed to OLED (2x2 pixel blocks)
  4. Overlay game characters if received:
     - Draw square at p1.x, p1.y for Player 1
     - Draw circle at p2.x, p2.y for Player 2
  5. Display frame on OLED
  6. Return camera frame buffer
          



            
Key Features:
            

              
ESP-NOW receive-only mode for game state packets
              
MAC address-based player identification (P1: 0xAC, P2: 0x9C)
              
Real-time camera frame capture and dithering
              
180° rotation for correct display orientation
              
AR overlay of game characters on live camera feed
              
Automatic camera ID detection from MAC address
            
          


            

              
                
                View CircleSquareCamera.ino
              
            
            

              
                
                Download CircleSquareCamera.zip
              
            
            

              
                
                View camera_pins.h
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        


        

          
Game Board Code (TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino)
          

            Extended version of the CircleSquareShapeSongSwingalong game that broadcasts player state packets to both peer game boards and camera boards via ESP-NOW. This enables real-time AR visualization of gameplay on camera displays.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, OLED display
  2. Set WiFi mode to STA
  3. Read self MAC address
  4. Initialize ESP-NOW
  5. Identify player role (P1 or P2) from MAC address
  6. Add peers: peer game board + Camera A + Camera B
  7. Register ESP-NOW send/receive callbacks
  8. Initialize touch sensors (warm-up readings)
  9. Reset player states (left/right positioning)

LOOP:
  1. Update touch sensor readings
  2. Process player input (left, right, jump)
  3. Update player physics (movement, gravity, collision)
  4. Create NetPacket with player state and ready flag
  5. Broadcast packet to all peers:
     - Other game board (peerMac)
     - Camera A (camA_MAC)
     - Camera B (camB_MAC)
  6. Render game state to OLED:
     - Draw ground line
     - Draw own character (square if P1, circle if P2)
     - Draw other player character
     - Display "Waiting Player" if other not ready
  7. Delay for frame timing (30ms)
          



            
Key Features:
            

              
ESP-NOW one-to-many broadcast to game boards and camera boards
              
MAC address-based player identification (P1: D8:3B:DA:75:05:AC, P2: D8:3B:DA:75:E1:9C)
              
Touch-based input controls with threshold detection
              
Real-time physics simulation (movement, gravity, collision)
              
Ready state synchronization for coordinated gameplay start
              
Game state rendering with square/circle character representation
            
          


            
What You Need to Know:
            

              This code extends the original CircleSquareShapeSongSwingalong game by adding camera board peers to the ESP-NOW network. The game broadcasts player state packets (position, velocity, onGround flag) to enable AR overlay on camera displays. The camera boards use these packets to draw game characters at the correct positions relative to the camera feed, creating an augmented reality experience. Player identification is done via MAC address comparison, with P1 using square representation and P2 using circle representation. See Ethical AI Use section for ChatGPT co-development transcript.
            
          


            

              
                
                View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino
              
            
            

              
                
                Download TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.zip
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        
      
    


    

      
Reflections & Learnings
      
Key insights and learnings from working with networking and communications protocols.
    


    

      
Contributions
      
Acknowledgements and team roles for networking and communications work.
      

        

          ESP-NOW Multiplayer Game
          
Developed CircleSquareShapeSongSwingalong multiplayer game with ESP-NOW wireless communication during Week 2. See Week 2 documentation for details.
        
        

          Wi-Fi Camera Streaming System
          
Developed camera livestreaming and Edge AI face detection system during Week 10. See Week 10 documentation for details.
        
      
    


    

      
Ethical AI Use
      
Transparent documentation of AI assistance used in this week's networking and communications work.


        
AI-Assisted Camera Integration Game Update
        

          Used ChatGPT for collaborative development of the CircleSquareCamera augmented reality system, integrating ESP-NOW communication between game boards and camera boards. The AI assisted with ESP-NOW packet structure design, camera feed dithering algorithms, AR overlay implementation, and real-time synchronization of player states with camera displays. This extends the original CircleSquareShapeSongSwingalong multiplayer game from Week 2 with augmented reality visualization.
        
        

          
ChatGPT Co-Development Transcript:
          

            
              
              View ChatGPT Transcript
            
          
        
      


        
AI-Assisted Week 11 Page Creation and Assignment Documentation
        

          Cursor AI assisted with creating the complete Week 11 documentation page, including networking protocols overview, group assignment CircleSquareCamera AR documentation, individual assignment sections summarizing ESP-NOW multiplayer game and Wi-Fi camera streaming systems, network address tables, and all formatting and styling. The AI helped refine and organize content from Week 2 and Week 10 into professional, succinct networking documentation while preserving key technical details.
        
        

          The AI was used to:
        
        

          
Create week11.html from the week8.html template structure
          
Populate networking and communications resources from the MIT Academy class page
          
Summarize and integrate content from Week 2 (ESP-NOW multiplayer game) and Week 10 (Wi-Fi camera streaming)
          
Create network address tables with MAC and IP addresses for both networked systems
          
Format and style all sections for professional presentation
          
Embed images, videos, and design file download links
        
        

          
Complete AI Assistance Transcript:
          

            
              
              View HTML Transcript
            
            
              
              Download Markdown
            
          
        
      
    


      

        

          
            
          
          This work is licensed under a
          
            Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
          
        
      
    


  

  
  
  
  
  


  

    

      ← Back to Home
    


      

        
      
      

        
Week 11
        
Networking & Communications
      
      
    


    

      

        
        Week Highlights
      
      
Key captures from the group and individual assignments with links to their sections.


      

        
        

          
ESP-NOW Multiplayer Network
          
            
          
        
        
        

          
Wi-Fi Camera Livestream
          
            
          
        
        
        

          
CircleSquareCamera AR
          
            
          
        
      
    


    

      

        
        Table of Contents
      
      

        

          
Course Content
          

            
• Week Highlights
            
• Intro
            
• This Week's Goals
            
• Assignments
            
• Tools & Materials
          
        
        

          
Training & Documentation
          

            
• Training Documentation
            
• Recitation Notes
            
• Useful Documentation
            
• Networking Protocols
          
        
        

          
Assignments & Projects
          

            
• Group Assignment
            
• Individual Assignment
            
• Design Files
            
• Reflections & Learnings
            
• Contributions
            
• Ethical AI Use
          
        
      
    


    

      

        
Networking & Communications
        
This week focuses on connecting devices through wired and wireless networks. We'll explore communication protocols, network addressing, and how to build networked systems with local input and output devices.
        

          
Understanding wired and wireless communication protocols
          
Learning network addressing and bus architectures
          
Implementing networked nodes with input and output devices
        
      
      

        
This Week's Goals
        

          
Network Protocols
Learn about ESP-NOW, Wi-Fi, I²C, SPI, UART, and other communication protocols
          
Network Addressing
Understand MAC addresses, IP addresses, and bus addressing schemes
          
Networked Systems
Design, build, and connect wired or wireless nodes with local I/O devices
        
      
    


    

      

        
Assignments
        

          
Group Assignment
Send a message between two projects
          
Individual Assignment
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s)
        
      
      

        
Tools & Materials
        

          
ESP32-S3 Microcontroller Boards
          
Wi-Fi Access Point
          
OLED Displays (SSD1306)
          
Camera Modules
          
Touch Sensors
        
      
    


    

      
Training Documentation
      
Training materials and documentation for networking and communications protocols.


      

        
Recitation Notes
        

          Networking and communications recitation covered ESP-NOW protocol implementation, MAC address-based device identification, peer-to-peer communication setup, and network address configuration for embedded systems.
        
      


      

        
Networking Protocols Overview
        

          Networking and communications enable devices to exchange data through wired and wireless connections. Understanding different protocols helps select the right communication method for each application.
        

Wired Communication
        

          
UART/USART — Asynchronous serial communication (RS-232, RS-422, RS-485)
          
I²C/TWI — Two-wire interface with SCL/SDA, requires pull-up resistors
          
SPI — Serial Peripheral Interface with MISO/MOSI/CS/SCK lines
          
USB — Universal Serial Bus for device communication
          
Ethernet — Wired network communication
        

Wireless Communication
        

          
ESP-NOW — Low-power peer-to-peer communication for ESP32 devices
          
Wi-Fi — 802.11 wireless networking (2.4 GHz, 5.8 GHz)
          
Bluetooth/BLE — Short-range wireless communication
          
LoRa — Long-range, low-power wireless communication
          
RF — Radio frequency communication (433 MHz, 868 MHz, 915 MHz, 2.4 GHz)
        

Network Addressing
        

          Devices on networks require unique addresses for identification and communication:
        
        

          
MAC Address — Media Access Control address, unique hardware identifier (e.g., D8:3B:DA:75:05:AC)
          
IP Address — Internet Protocol address for network routing (e.g., 192.168.1.100)
          
Bus Address — Device address on shared communication buses (I²C, SPI)
        
      
    


    

      
Useful Documentation
      
Essential resources for networking and communications protocols.


        

          
Class Page
          
Networking and Communications - MIT Academy
          
Comprehensive resource covering wired protocols (UART, I²C, SPI, USB, Ethernet), wireless protocols (Wi-Fi, Bluetooth, LoRa, RF), network addressing, OSI layers, modulation techniques, channel sharing methods, error detection/correction, and Internet protocols. Includes detailed tutorials, code examples, and implementation guides for various communication protocols.
        
        

          
Key Topics
          

            
Wired Communication: UART, I²C, SPI, USB, Ethernet, CAN, LIN, MODBUS
            
Wireless Communication: ESP-NOW, Wi-Fi (802.11), Bluetooth/BLE, LoRa, RF (433/868/915 MHz, 2.4/5.8 GHz)
            
Network Protocols: IPv4/IPv6, TCP/UDP, HTTP, DNS, DHCP, NAT
            
Modulation: PCM, PPM, OOK, FSK, BPSK, QAM, OFDM, FHSS, DSSS, UWB
            
Channel Sharing: TDMA, FDMA, CSMA/CD, CSMA/CA, CDMA, MIMO
          
        
      
    


    

      
Group Assignment: Send a Message Between Two Projects
      
Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols.


        
Assignment Description
        

          Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols. The CircleSquareCamera Augmented Reality system extends the ESP-NOW multiplayer game from Week 2 by integrating camera boards that receive game state updates and overlay player positions on live camera feeds.
        
      


      

        

          
          CircleSquareCamera Augmented Reality
        


          An augmented reality system that combines the ESP-NOW multiplayer game CircleSquareShapeSongSwingalong (documented in Week 2 and Week 11 individual assignment) with camera subsystems from the final project. The system enables game boards to send player state updates via ESP-NOW to camera boards, which overlay game characters (square and circle) on live camera feeds displayed on OLED screens. Camera code development spans Week 7, Week 8, Final Project, and Week 12 (placeholder link).
        


          

            
            
          
          

            

              Development Note: Initial implementation attempted to send camera data from camera boards to game boards. After two hours of troubleshooting communication issues, the architecture was reversed: the final system sends game state packets from game boards to camera boards via ESP-NOW. This approach proved more efficient for real-time AR overlay visualization, as game state is lightweight compared to camera frame data.
            
          
          

            
              
            
            
          
        

Design & Build
        

          The system consists of two networked subsystems: game boards (XIAO ESP32-S3 with touch sensors and OLED displays) and camera boards (XIAO ESP32-S3 Sense with integrated cameras and OLED displays). Game boards run the CircleSquareShapeSongSwingalong multiplayer game, processing touch input, updating player physics, and broadcasting player state packets via ESP-NOW to both the other game board and all camera boards. Camera boards receive these packets, capture camera frames, process them through Floyd–Steinberg dithering for OLED display, and overlay game character positions (square for Player 1, circle for Player 2) on the live feed.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification for game boards and camera boards

          Communication Pattern: One-to-many broadcast from game boards to peer game board and all camera boards
        

Input & Output Devices
        

          
Game Boards:
            

              
Input: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering
            
          
          
Camera Boards:
            

              
Input: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, ESP-NOW packets for game state
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for displaying dithered camera feed with AR overlay
            
          
        

Connections
        

          Game Boards: Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. ESP-NOW communication is handled through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.


          Camera Boards: The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. ESP-NOW reception uses the ESP32-S3's built-in Wi-Fi radio in receive-only mode.
        


        

          
Network Address Tables
          

            Complete network addressing information for all devices in the CircleSquareCamera AR system, including MAC addresses (hardware identifiers) for ESP-NOW communication.
          

ESP-NOW Network (CircleSquareCamera AR System)
          

            

              

                
Device
                
Component
                
MAC Address
                
Network Type
                
Input Device
                
Output Device
              
            
            

              

                
P1
                
XIAO ESP32-S3
                
D8:3B:DA:75:05:AC
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
P2
                
XIAO ESP32-S3
                
D8:3B:DA:75:E1:9C
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
Camera A
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:E2:C0
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
              

                
Camera B
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:D6:38
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
            
          


            

              Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. Game boards broadcast player state packets to both the peer game board and all camera boards. Camera boards operate in receive-only mode, processing incoming ESP-NOW packets to identify player positions and overlay them on camera feeds.
            
          
        


          
Download Design Files
          

            Complete Arduino code for game boards and camera boards, including ESP-NOW communication, camera feed processing, and AR overlay implementation. See Design Files section for detailed documentation and download links.
          
        
      
    


    

      
Individual Assignment: Networking & Communications
      
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s). This assignment demonstrates two networked systems: an ESP-NOW multiplayer game and a Wi-Fi camera streaming system with edge AI processing.


        
Related Documentation
        

          
Week 2: Embedded Programming & Nintendo Training — ESP-NOW multiplayer game development
          
Week 2: Design Files — CircleSquareShapeSongSwingalong Arduino code
          
Week 10: Camera & Edge AI — Wi-Fi camera streaming system
          
Week 10: Design Files - Camera — Camera stream code and system implementation
        
      


      

        
Networked System 1: CircleSquareShapeSongSwingalong (ESP-NOW)
        

          A collaborative multiplayer game featuring geometric shape interactions, synchronized gameplay between two ESP32-S3 devices using ESP-NOW wireless communication. The system uses MAC address-based player identification and real-time state synchronization for coordinated gameplay. See Week 2 documentation and design files for complete details.
        

Design & Build
        

          The system consists of two XIAO ESP32-S3 development boards, each with an SSD1306 OLED display (128x64) and capacitive touch sensors. The boards communicate wirelessly using ESP-NOW protocol, enabling low-latency peer-to-peer communication without requiring a Wi-Fi access point.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification
        

Input & Output Devices
        

          
Input Devices: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
          
Output Devices: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering and player state visualization
        


          

            

              
            
            
Shape Song Swing Along educational content integration and gameplay
          
          

            

              
            
            
Circle and square shape interaction gameplay mechanics
          
        

Connections
        

          Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. The ESP-NOW communication is handled entirely through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.
        


          
Download Design Files
          

            
TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino — Complete Arduino sketch for ESP-NOW multiplayer game
            
get_mac_address.ino — Utility to read MAC addresses from ESP32-S3 devices
          
        
      


      

        
Networked System 2: Camera Livestream & Edge AI (Wi-Fi)
        

          A Wi-Fi-enabled camera system using ESP32-S3's built-in camera and HTTP server capabilities to stream JPEG frames over Wi-Fi using MJPEG (Motion JPEG) protocol. The system also includes edge AI face detection using a FOMO (Faster Objects, More Objects) model from Edge Impulse for real-time person tracking. See Week 10 documentation and design files for complete details.
        

Design & Build
        

          The system uses a XIAO ESP32-S3 Sense development board with an integrated camera module. The camera captures frames at QQVGA (160x120) or QVGA resolution, processes them through an Edge AI inference pipeline, and streams the results over Wi-Fi via HTTP multipart response. The system can also display camera output and inference results on an OLED display connected via I²C.
        

Network Type & Protocol
        

          Network Type: Wireless local area network (Wi-Fi)

          Protocol: HTTP over Wi-Fi (802.11), MJPEG streaming

          Addressing Scheme: IP address (DHCP-assigned) and MAC address (hardware identifier)
        

Input & Output Devices
        

          
Input Devices: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, button for capture trigger
          
Output Devices: OLED display (SSD1306, 128x64, I²C address 0x3C) for displaying camera frames and inference results, Wi-Fi HTTP server for remote streaming
        


          

            
            
          
        
        

          

            
            
          
          

            
            
          
          

            
            
          
        

Connections
        

          The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. Wi-Fi connection is established through the ESP32-S3's built-in Wi-Fi radio.
        


          
Download Design Files
          

            
camera_stream.zip — Complete camera stream project (includes .ino and .h files)
            
camera_stream.ino — Main Arduino sketch for camera livestreaming
            
camera_pins.h — GPIO pin definitions for XIAO ESP32-S3 camera module
            
ei-face-detection--fomo-arduino-1.0.90.zip — Edge Impulse Arduino library for FOMO-based face detection
          
        
      


      

        
Network Address Tables
        

          Complete network addressing information for all connected devices, including MAC addresses (hardware identifiers) and IP addresses (network identifiers) for Wi-Fi-connected devices.
        

ESP-NOW Network (CircleSquareShapeSongSwingalong)
        

          

            

              
Device
              
Component
              
MAC Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Player 1
              
XIAO ESP32-S3
              
D8:3B:DA:75:05:AC
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
            

              
Player 2
              
XIAO ESP32-S3
              
D8:3B:DA:75:E1:9C
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
          
        

Wi-Fi Network (Camera Livestream System)
        

          

            

              
Device
              
Component
              
MAC Address
              
IP Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Camera Node
              
XIAO ESP32-S3 Sense
              
[Device MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
Camera module, Button
              
OLED (SSD1306, 0x3C), HTTP Server
            
            

              
Client Device
              
Web Browser
              
[Client MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
N/A
              
Display (receives MJPEG stream)
            
          
        


          

            Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. IP addresses are assigned dynamically via DHCP when connecting to a Wi-Fi network. For ESP-NOW communication, only MAC addresses are used (no IP addresses required). The camera system's IP address is printed to the Serial Monitor upon Wi-Fi connection establishment.
          
        
      
    


    

      
Design Files
      
Complete design files for networked systems including ESP-NOW multiplayer game and Wi-Fi camera streaming implementations with Arduino firmware and configuration files.


      

        
ESP-NOW Multiplayer Game (CircleSquareShapeSongSwingalong)
        

          Arduino code for ESP-NOW wireless multiplayer game with MAC address-based player identification and synchronized game state management.
        
        

          
Key Features:
          

            
ESP-NOW wireless peer-to-peer communication
            
MAC address-based player identification (D8:3B:DA:75:05:AC and D8:3B:DA:75:E1:9C)
            
Real-time game state synchronization
            
Ready state coordination between players
            
Touch-based input controls (left, right, jump)
            
OLED display output for game rendering
          
        


          

            
              
              View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino
            
          
          

            
              
              Download .ino
            
          
          

            
              
              Download get_mac_address.ino
            
          
        
      


      

        
Wi-Fi Camera Livestream & Edge AI
        

          Arduino code for ESP32-S3 camera livestreaming over Wi-Fi using MJPEG protocol and Edge AI face detection using FOMO models from Edge Impulse.
        


          
Camera Livestream Pseudocode:
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Configure camera pins (from camera_pins.h)
  3. Create camera_config_t structure
  4. Initialize camera with esp_camera_init()
  5. Connect to Wi-Fi network
  6. Start HTTP server with stream handler

STREAM_HANDLER:
  1. Set HTTP response type to "multipart/x-mixed-replace"
  2. Enter loop: capture frame, send via HTTP, repeat

LOOP:
  - Minimal delay to allow other tasks
        



          
Key Features:
          

            
Wi-Fi HTTP server for MJPEG streaming
            
Edge AI face detection using FOMO models
            
Real-time camera frame capture and processing
            
OLED display output for local visualization
            
IP address assignment via DHCP
            
MAC address for hardware identification
          
        


          

            
              
              View camera_stream.ino
            
          
          

            
              
              Download camera_stream.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              Download Edge AI Library
            
          
        
      


      

        
Video Dithering Stream to OLED
        

          Arduino code for capturing camera frames and streaming them to an OLED display using Floyd–Steinberg dithering for high-quality monochrome rendering. This code demonstrates real-time image processing and display optimization for low-resolution OLED screens.
        


          
How the Code Works (Pseudocode):
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Initialize I²C bus for OLED display
  3. Initialize OLED display (128x64, I²C address 0x3C)
  4. Configure camera pins using camera_pins.h definitions
  5. Initialize camera with QQVGA resolution (160x120)
  6. Set pixel format to GRAYSCALE
  7. Display initialization status on OLED

LOOP:
  1. Capture camera frame using esp_camera_fb_get()
  2. Process frame through showDitheredPreview():
     a. Downsample camera frame (160x120) to OLED resolution (128x64)
     b. Average pixel values in each downsampled region
     c. Store results in gray_buffer array
     d. Normalize brightness values to full range
     e. Apply Floyd–Steinberg dithering algorithm
     f. Render dithered result to OLED display
  3. Return camera frame buffer using esp_camera_fb_return()
  4. Repeat continuously for live preview

DITHERING ALGORITHM (Floyd–Steinberg):
  For each pixel from top-left to bottom-right:
    1. Quantize current pixel (0 or 255)
    2. Calculate quantization error
    3. Distribute error to neighboring pixels:
       - Right: 7/16 of error
       - Bottom-left: 3/16 of error
       - Bottom: 5/16 of error
       - Bottom-right: 1/16 of error
        



          
Key Features:
          

            
Real-time camera frame capture at QQVGA resolution (160x120)
            
Grayscale image processing for efficient monochrome display
            
Floyd–Steinberg dithering algorithm for high-quality visual output
            
Automatic brightness normalization for optimal contrast
            
Continuous live preview stream to OLED display
            
Optimized downsampling from camera resolution to OLED resolution (128x64)
          
        


          
What You Need to Know:
          

            This code provides a foundation for displaying camera feeds on OLED displays. The Floyd–Steinberg dithering algorithm distributes quantization errors across neighboring pixels, creating smooth gradients and improved visual quality compared to simple thresholding. The code uses grayscale format for efficient processing, and the downsampling step averages multiple camera pixels into each OLED pixel to maintain image clarity.
          
          

            This implementation serves as a building block for the CircleSquareCamera AR system, where camera feeds are combined with game state overlays. See Ethical AI Use section for ChatGPT co-development transcript.
          
        


          

            
              
              View view_camera_oled.ino
            
          
          

            
              
              Download view_camera_oled.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              View ChatGPT Transcript
            
          
        
      


      

        
Group Assignment: CircleSquareCamera Augmented Reality System
        

          Complete Arduino code for the CircleSquareCamera AR system, including game board code that broadcasts player states via ESP-NOW, and camera board code that receives game packets and overlays player positions on live camera feeds.
        


        

          
Camera Board Code (CircleSquareCamera.ino)
          

            Camera board receives ESP-NOW packets from game boards, captures camera frames, processes them through dithering, and overlays game character positions (square for P1, circle for P2) on the live feed displayed on OLED.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, I²C, OLED display
  2. Initialize ESP-NOW in receive-only mode
  3. Register ESP-NOW receive callback
  4. Initialize camera (QQVGA, GRAYSCALE)
  5. Determine camera ID from MAC address (A or B)
  6. Display camera ready status

ESPNOW_RECEIVE_CALLBACK:
  1. Check if packet is correct size (NetPacket)
  2. Extract player state from packet
  3. Identify player (P1 or P2) based on sender MAC address
  4. Store player state in p1 or p2 variable
  5. Set gotP1 or gotP2 flag

LOOP:
  1. Capture camera frame (esp_camera_fb_get)
  2. Process frame through makeFrame():
     a. Downsample camera (160x120) to 64x32
     b. Normalize brightness
     c. Apply Floyd–Steinberg dithering
     d. Rotate 180° for correct orientation
  3. Draw camera feed to OLED (2x2 pixel blocks)
  4. Overlay game characters if received:
     - Draw square at p1.x, p1.y for Player 1
     - Draw circle at p2.x, p2.y for Player 2
  5. Display frame on OLED
  6. Return camera frame buffer
          



            
Key Features:
            

              
ESP-NOW receive-only mode for game state packets
              
MAC address-based player identification (P1: 0xAC, P2: 0x9C)
              
Real-time camera frame capture and dithering
              
180° rotation for correct display orientation
              
AR overlay of game characters on live camera feed
              
Automatic camera ID detection from MAC address
            
          


            

              
                
                View CircleSquareCamera.ino
              
            
            

              
                
                Download CircleSquareCamera.zip
              
            
            

              
                
                View camera_pins.h
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        


        

          
Game Board Code (TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino)
          

            Extended version of the CircleSquareShapeSongSwingalong game that broadcasts player state packets to both peer game boards and camera boards via ESP-NOW. This enables real-time AR visualization of gameplay on camera displays.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, OLED display
  2. Set WiFi mode to STA
  3. Read self MAC address
  4. Initialize ESP-NOW
  5. Identify player role (P1 or P2) from MAC address
  6. Add peers: peer game board + Camera A + Camera B
  7. Register ESP-NOW send/receive callbacks
  8. Initialize touch sensors (warm-up readings)
  9. Reset player states (left/right positioning)

LOOP:
  1. Update touch sensor readings
  2. Process player input (left, right, jump)
  3. Update player physics (movement, gravity, collision)
  4. Create NetPacket with player state and ready flag
  5. Broadcast packet to all peers:
     - Other game board (peerMac)
     - Camera A (camA_MAC)
     - Camera B (camB_MAC)
  6. Render game state to OLED:
     - Draw ground line
     - Draw own character (square if P1, circle if P2)
     - Draw other player character
     - Display "Waiting Player" if other not ready
  7. Delay for frame timing (30ms)
          



            
Key Features:
            

              
ESP-NOW one-to-many broadcast to game boards and camera boards
              
MAC address-based player identification (P1: D8:3B:DA:75:05:AC, P2: D8:3B:DA:75:E1:9C)
              
Touch-based input controls with threshold detection
              
Real-time physics simulation (movement, gravity, collision)
              
Ready state synchronization for coordinated gameplay start
              
Game state rendering with square/circle character representation
            
          


            
What You Need to Know:
            

              This code extends the original CircleSquareShapeSongSwingalong game by adding camera board peers to the ESP-NOW network. The game broadcasts player state packets (position, velocity, onGround flag) to enable AR overlay on camera displays. The camera boards use these packets to draw game characters at the correct positions relative to the camera feed, creating an augmented reality experience. Player identification is done via MAC address comparison, with P1 using square representation and P2 using circle representation. See Ethical AI Use section for ChatGPT co-development transcript.
            
          


            

              
                
                View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino
              
            
            

              
                
                Download TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.zip
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        
      
    


    

      
Reflections & Learnings
      
Key insights and learnings from working with networking and communications protocols.
    


    

      
Contributions
      
Acknowledgements and team roles for networking and communications work.
      

        

          ESP-NOW Multiplayer Game
          
Developed CircleSquareShapeSongSwingalong multiplayer game with ESP-NOW wireless communication during Week 2. See Week 2 documentation for details.
        
        

          Wi-Fi Camera Streaming System
          
Developed camera livestreaming and Edge AI face detection system during Week 10. See Week 10 documentation for details.
        
      
    


    

      
Ethical AI Use
      
Transparent documentation of AI assistance used in this week's networking and communications work.


        
AI-Assisted Camera Integration Game Update
        

          Used ChatGPT for collaborative development of the CircleSquareCamera augmented reality system, integrating ESP-NOW communication between game boards and camera boards. The AI assisted with ESP-NOW packet structure design, camera feed dithering algorithms, AR overlay implementation, and real-time synchronization of player states with camera displays. This extends the original CircleSquareShapeSongSwingalong multiplayer game from Week 2 with augmented reality visualization.
        
        

          
ChatGPT Co-Development Transcript:
          

            
              
              View ChatGPT Transcript
            
          
        
      


        
AI-Assisted Week 11 Page Creation and Assignment Documentation
        

          Cursor AI assisted with creating the complete Week 11 documentation page, including networking protocols overview, group assignment CircleSquareCamera AR documentation, individual assignment sections summarizing ESP-NOW multiplayer game and Wi-Fi camera streaming systems, network address tables, and all formatting and styling. The AI helped refine and organize content from Week 2 and Week 10 into professional, succinct networking documentation while preserving key technical details.
        
        

          The AI was used to:
        
        

          
Create week11.html from the week8.html template structure
          
Populate networking and communications resources from the MIT Academy class page
          
Summarize and integrate content from Week 2 (ESP-NOW multiplayer game) and Week 10 (Wi-Fi camera streaming)
          
Create network address tables with MAC and IP addresses for both networked systems
          
Format and style all sections for professional presentation
          
Embed images, videos, and design file download links
        
        

          
Complete AI Assistance Transcript:
          

            
              
              View HTML Transcript
            
            
              
              Download Markdown
            
          
        
      
    


      

        

          
            
          
          This work is licensed under a
          
            Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
          
        
      
    


  

  
  
  
  
  


  

    

      ← Back to Home
    


      

        
      
      

        
Week 11
        
Networking & Communications
      
      
    


    

      

        
        Week Highlights
      
      
Key captures from the group and individual assignments with links to their sections.


      

        
        

          
ESP-NOW Multiplayer Network
          
            
          
        
        
        

          
Wi-Fi Camera Livestream
          
            
          
        
        
        

          
CircleSquareCamera AR
          
            
          
        
      
    


    

      

        
        Table of Contents
      
      

        

          
Course Content
          

            
• Week Highlights
            
• Intro
            
• This Week's Goals
            
• Assignments
            
• Tools & Materials
          
        
        

          
Training & Documentation
          

            
• Training Documentation
            
• Recitation Notes
            
• Useful Documentation
            
• Networking Protocols
          
        
        

          
Assignments & Projects
          

            
• Group Assignment
            
• Individual Assignment
            
• Design Files
            
• Reflections & Learnings
            
• Contributions
            
• Ethical AI Use
          
        
      
    


    

      

        
Networking & Communications
        
This week focuses on connecting devices through wired and wireless networks. We'll explore communication protocols, network addressing, and how to build networked systems with local input and output devices.
        

          
Understanding wired and wireless communication protocols
          
Learning network addressing and bus architectures
          
Implementing networked nodes with input and output devices
        
      
      

        
This Week's Goals
        

          
Network Protocols
Learn about ESP-NOW, Wi-Fi, I²C, SPI, UART, and other communication protocols
          
Network Addressing
Understand MAC addresses, IP addresses, and bus addressing schemes
          
Networked Systems
Design, build, and connect wired or wireless nodes with local I/O devices
        
      
    


    

      

        
Assignments
        

          
Group Assignment
Send a message between two projects
          
Individual Assignment
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s)
        
      
      

        
Tools & Materials
        

          
ESP32-S3 Microcontroller Boards
          
Wi-Fi Access Point
          
OLED Displays (SSD1306)
          
Camera Modules
          
Touch Sensors
        
      
    


    

      
Training Documentation
      
Training materials and documentation for networking and communications protocols.


      

        
Recitation Notes
        

          Networking and communications recitation covered ESP-NOW protocol implementation, MAC address-based device identification, peer-to-peer communication setup, and network address configuration for embedded systems.
        
      


      

        
Networking Protocols Overview
        

          Networking and communications enable devices to exchange data through wired and wireless connections. Understanding different protocols helps select the right communication method for each application.
        

Wired Communication
        

          
UART/USART — Asynchronous serial communication (RS-232, RS-422, RS-485)
          
I²C/TWI — Two-wire interface with SCL/SDA, requires pull-up resistors
          
SPI — Serial Peripheral Interface with MISO/MOSI/CS/SCK lines
          
USB — Universal Serial Bus for device communication
          
Ethernet — Wired network communication
        

Wireless Communication
        

          
ESP-NOW — Low-power peer-to-peer communication for ESP32 devices
          
Wi-Fi — 802.11 wireless networking (2.4 GHz, 5.8 GHz)
          
Bluetooth/BLE — Short-range wireless communication
          
LoRa — Long-range, low-power wireless communication
          
RF — Radio frequency communication (433 MHz, 868 MHz, 915 MHz, 2.4 GHz)
        

Network Addressing
        

          Devices on networks require unique addresses for identification and communication:
        
        

          
MAC Address — Media Access Control address, unique hardware identifier (e.g., D8:3B:DA:75:05:AC)
          
IP Address — Internet Protocol address for network routing (e.g., 192.168.1.100)
          
Bus Address — Device address on shared communication buses (I²C, SPI)
        
      
    


    

      
Useful Documentation
      
Essential resources for networking and communications protocols.


        

          
Class Page
          
Networking and Communications - MIT Academy
          
Comprehensive resource covering wired protocols (UART, I²C, SPI, USB, Ethernet), wireless protocols (Wi-Fi, Bluetooth, LoRa, RF), network addressing, OSI layers, modulation techniques, channel sharing methods, error detection/correction, and Internet protocols. Includes detailed tutorials, code examples, and implementation guides for various communication protocols.
        
        

          
Key Topics
          

            
Wired Communication: UART, I²C, SPI, USB, Ethernet, CAN, LIN, MODBUS
            
Wireless Communication: ESP-NOW, Wi-Fi (802.11), Bluetooth/BLE, LoRa, RF (433/868/915 MHz, 2.4/5.8 GHz)
            
Network Protocols: IPv4/IPv6, TCP/UDP, HTTP, DNS, DHCP, NAT
            
Modulation: PCM, PPM, OOK, FSK, BPSK, QAM, OFDM, FHSS, DSSS, UWB
            
Channel Sharing: TDMA, FDMA, CSMA/CD, CSMA/CA, CDMA, MIMO
          
        
      
    


    

      
Group Assignment: Send a Message Between Two Projects
      
Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols.


        
Assignment Description
        

          Send a message between two projects. This assignment demonstrates inter-project communication using wired or wireless networking protocols. The CircleSquareCamera Augmented Reality system extends the ESP-NOW multiplayer game from Week 2 by integrating camera boards that receive game state updates and overlay player positions on live camera feeds.
        
      


      

        

          
          CircleSquareCamera Augmented Reality
        


          An augmented reality system that combines the ESP-NOW multiplayer game CircleSquareShapeSongSwingalong (documented in Week 2 and Week 11 individual assignment) with camera subsystems from the final project. The system enables game boards to send player state updates via ESP-NOW to camera boards, which overlay game characters (square and circle) on live camera feeds displayed on OLED screens. Camera code development spans Week 7, Week 8, Final Project, and Week 12 (placeholder link).
        


          

            
            
          
          

            

              Development Note: Initial implementation attempted to send camera data from camera boards to game boards. After two hours of troubleshooting communication issues, the architecture was reversed: the final system sends game state packets from game boards to camera boards via ESP-NOW. This approach proved more efficient for real-time AR overlay visualization, as game state is lightweight compared to camera frame data.
            
          
          

            
              
            
            
          
        

Design & Build
        

          The system consists of two networked subsystems: game boards (XIAO ESP32-S3 with touch sensors and OLED displays) and camera boards (XIAO ESP32-S3 Sense with integrated cameras and OLED displays). Game boards run the CircleSquareShapeSongSwingalong multiplayer game, processing touch input, updating player physics, and broadcasting player state packets via ESP-NOW to both the other game board and all camera boards. Camera boards receive these packets, capture camera frames, process them through Floyd–Steinberg dithering for OLED display, and overlay game character positions (square for Player 1, circle for Player 2) on the live feed.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification for game boards and camera boards

          Communication Pattern: One-to-many broadcast from game boards to peer game board and all camera boards
        

Input & Output Devices
        

          
Game Boards:
            

              
Input: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering
            
          
          
Camera Boards:
            

              
Input: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, ESP-NOW packets for game state
              
Output: SSD1306 OLED display (128x64, I²C address 0x3C) for displaying dithered camera feed with AR overlay
            
          
        

Connections
        

          Game Boards: Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. ESP-NOW communication is handled through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.


          Camera Boards: The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. ESP-NOW reception uses the ESP32-S3's built-in Wi-Fi radio in receive-only mode.
        


        

          
Network Address Tables
          

            Complete network addressing information for all devices in the CircleSquareCamera AR system, including MAC addresses (hardware identifiers) for ESP-NOW communication.
          

ESP-NOW Network (CircleSquareCamera AR System)
          

            

              

                
Device
                
Component
                
MAC Address
                
Network Type
                
Input Device
                
Output Device
              
            
            

              

                
P1
                
XIAO ESP32-S3
                
D8:3B:DA:75:05:AC
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
P2
                
XIAO ESP32-S3
                
D8:3B:DA:75:E1:9C
                
ESP-NOW (P2P)
                
Touch sensors (6 pads)
                
OLED (SSD1306, 0x3C)
              
              

                
Camera A
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:E2:C0
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
              

                
Camera B
                
XIAO ESP32-S3 Sense
                
B8:F8:62:F9:D6:38
                
ESP-NOW (Receive)
                
Camera module, ESP-NOW packets
                
OLED (SSD1306, 0x3C)
              
            
          


            

              Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. Game boards broadcast player state packets to both the peer game board and all camera boards. Camera boards operate in receive-only mode, processing incoming ESP-NOW packets to identify player positions and overlay them on camera feeds.
            
          
        


          
Download Design Files
          

            Complete Arduino code for game boards and camera boards, including ESP-NOW communication, camera feed processing, and AR overlay implementation. See Design Files section for detailed documentation and download links.
          
        
      
    


    

      
Individual Assignment: Networking & Communications
      
Design, build, and connect wired or wireless node(s) with network or bus addresses and local input &/or output device(s). This assignment demonstrates two networked systems: an ESP-NOW multiplayer game and a Wi-Fi camera streaming system with edge AI processing.


        
Related Documentation
        

          
Week 2: Embedded Programming & Nintendo Training — ESP-NOW multiplayer game development
          
Week 2: Design Files — CircleSquareShapeSongSwingalong Arduino code
          
Week 10: Camera & Edge AI — Wi-Fi camera streaming system
          
Week 10: Design Files - Camera — Camera stream code and system implementation
        
      


      

        
Networked System 1: CircleSquareShapeSongSwingalong (ESP-NOW)
        

          A collaborative multiplayer game featuring geometric shape interactions, synchronized gameplay between two ESP32-S3 devices using ESP-NOW wireless communication. The system uses MAC address-based player identification and real-time state synchronization for coordinated gameplay. See Week 2 documentation and design files for complete details.
        

Design & Build
        

          The system consists of two XIAO ESP32-S3 development boards, each with an SSD1306 OLED display (128x64) and capacitive touch sensors. The boards communicate wirelessly using ESP-NOW protocol, enabling low-latency peer-to-peer communication without requiring a Wi-Fi access point.
        

Network Type & Protocol
        

          Network Type: Wireless peer-to-peer (ESP-NOW)

          Protocol: ESP-NOW (Espressif's proprietary low-power wireless communication protocol)

          Addressing Scheme: MAC address-based device identification
        

Input & Output Devices
        

          
Input Devices: Capacitive touch sensors (6 touch pads per device) for player controls (left, right, jump)
          
Output Devices: SSD1306 OLED display (128x64, I²C address 0x3C) for game rendering and player state visualization
        


          

            

              
            
            
Shape Song Swing Along educational content integration and gameplay
          
          

            

              
            
            
Circle and square shape interaction gameplay mechanics
          
        

Connections
        

          Each ESP32-S3 board connects to an OLED display via I²C (SDA pin 5, SCL pin 6) and reads touch sensor inputs from GPIO pins. The ESP-NOW communication is handled entirely through the ESP32-S3's built-in Wi-Fi radio, requiring no external hardware connections.
        


          
Download Design Files
          

            
TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino — Complete Arduino sketch for ESP-NOW multiplayer game
            
get_mac_address.ino — Utility to read MAC addresses from ESP32-S3 devices
          
        
      


      

        
Networked System 2: Camera Livestream & Edge AI (Wi-Fi)
        

          A Wi-Fi-enabled camera system using ESP32-S3's built-in camera and HTTP server capabilities to stream JPEG frames over Wi-Fi using MJPEG (Motion JPEG) protocol. The system also includes edge AI face detection using a FOMO (Faster Objects, More Objects) model from Edge Impulse for real-time person tracking. See Week 10 documentation and design files for complete details.
        

Design & Build
        

          The system uses a XIAO ESP32-S3 Sense development board with an integrated camera module. The camera captures frames at QQVGA (160x120) or QVGA resolution, processes them through an Edge AI inference pipeline, and streams the results over Wi-Fi via HTTP multipart response. The system can also display camera output and inference results on an OLED display connected via I²C.
        

Network Type & Protocol
        

          Network Type: Wireless local area network (Wi-Fi)

          Protocol: HTTP over Wi-Fi (802.11), MJPEG streaming

          Addressing Scheme: IP address (DHCP-assigned) and MAC address (hardware identifier)
        

Input & Output Devices
        

          
Input Devices: Camera module (integrated on XIAO ESP32-S3 Sense) for image capture, button for capture trigger
          
Output Devices: OLED display (SSD1306, 128x64, I²C address 0x3C) for displaying camera frames and inference results, Wi-Fi HTTP server for remote streaming
        


          

            
            
          
        
        

          

            
            
          
          

            
            
          
          

            
            
          
        

Connections
        

          The camera module is integrated directly on the XIAO ESP32-S3 Sense board, connected via parallel data bus (Y2-Y9), control signals (XCLK, PCLK, VSYNC, HREF), and I²C interface (SIOD, SIOC). The OLED display connects via I²C (SDA pin 5, SCL pin 6) with pull-up resistors. Wi-Fi connection is established through the ESP32-S3's built-in Wi-Fi radio.
        


          
Download Design Files
          

            
camera_stream.zip — Complete camera stream project (includes .ino and .h files)
            
camera_stream.ino — Main Arduino sketch for camera livestreaming
            
camera_pins.h — GPIO pin definitions for XIAO ESP32-S3 camera module
            
ei-face-detection--fomo-arduino-1.0.90.zip — Edge Impulse Arduino library for FOMO-based face detection
          
        
      


      

        
Network Address Tables
        

          Complete network addressing information for all connected devices, including MAC addresses (hardware identifiers) and IP addresses (network identifiers) for Wi-Fi-connected devices.
        

ESP-NOW Network (CircleSquareShapeSongSwingalong)
        

          

            

              
Device
              
Component
              
MAC Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Player 1
              
XIAO ESP32-S3
              
D8:3B:DA:75:05:AC
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
            

              
Player 2
              
XIAO ESP32-S3
              
D8:3B:DA:75:E1:9C
              
ESP-NOW (P2P)
              
Touch sensors (6 pads)
              
OLED (SSD1306, 0x3C)
            
          
        

Wi-Fi Network (Camera Livestream System)
        

          

            

              
Device
              
Component
              
MAC Address
              
IP Address
              
Network Type
              
Input Device
              
Output Device
            
          
          

            

              
Camera Node
              
XIAO ESP32-S3 Sense
              
[Device MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
Camera module, Button
              
OLED (SSD1306, 0x3C), HTTP Server
            
            

              
Client Device
              
Web Browser
              
[Client MAC]
              
192.168.x.x (DHCP)
              
Wi-Fi (802.11)
              
N/A
              
Display (receives MJPEG stream)
            
          
        


          

            Note: MAC addresses are hardware identifiers unique to each ESP32-S3 device. IP addresses are assigned dynamically via DHCP when connecting to a Wi-Fi network. For ESP-NOW communication, only MAC addresses are used (no IP addresses required). The camera system's IP address is printed to the Serial Monitor upon Wi-Fi connection establishment.
          
        
      
    


    

      
Design Files
      
Complete design files for networked systems including ESP-NOW multiplayer game and Wi-Fi camera streaming implementations with Arduino firmware and configuration files.


      

        
ESP-NOW Multiplayer Game (CircleSquareShapeSongSwingalong)
        

          Arduino code for ESP-NOW wireless multiplayer game with MAC address-based player identification and synchronized game state management.
        
        

          
Key Features:
          

            
ESP-NOW wireless peer-to-peer communication
            
MAC address-based player identification (D8:3B:DA:75:05:AC and D8:3B:DA:75:E1:9C)
            
Real-time game state synchronization
            
Ready state coordination between players
            
Touch-based input controls (left, right, jump)
            
OLED display output for game rendering
          
        


          

            
              
              View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix.ino
            
          
          

            
              
              Download .ino
            
          
          

            
              
              Download get_mac_address.ino
            
          
        
      


      

        
Wi-Fi Camera Livestream & Edge AI
        

          Arduino code for ESP32-S3 camera livestreaming over Wi-Fi using MJPEG protocol and Edge AI face detection using FOMO models from Edge Impulse.
        


          
Camera Livestream Pseudocode:
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Configure camera pins (from camera_pins.h)
  3. Create camera_config_t structure
  4. Initialize camera with esp_camera_init()
  5. Connect to Wi-Fi network
  6. Start HTTP server with stream handler

STREAM_HANDLER:
  1. Set HTTP response type to "multipart/x-mixed-replace"
  2. Enter loop: capture frame, send via HTTP, repeat

LOOP:
  - Minimal delay to allow other tasks
        



          
Key Features:
          

            
Wi-Fi HTTP server for MJPEG streaming
            
Edge AI face detection using FOMO models
            
Real-time camera frame capture and processing
            
OLED display output for local visualization
            
IP address assignment via DHCP
            
MAC address for hardware identification
          
        


          

            
              
              View camera_stream.ino
            
          
          

            
              
              Download camera_stream.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              Download Edge AI Library
            
          
        
      


      

        
Video Dithering Stream to OLED
        

          Arduino code for capturing camera frames and streaming them to an OLED display using Floyd–Steinberg dithering for high-quality monochrome rendering. This code demonstrates real-time image processing and display optimization for low-resolution OLED screens.
        


          
How the Code Works (Pseudocode):
          
SETUP:
  1. Initialize Serial communication (115200 baud)
  2. Initialize I²C bus for OLED display
  3. Initialize OLED display (128x64, I²C address 0x3C)
  4. Configure camera pins using camera_pins.h definitions
  5. Initialize camera with QQVGA resolution (160x120)
  6. Set pixel format to GRAYSCALE
  7. Display initialization status on OLED

LOOP:
  1. Capture camera frame using esp_camera_fb_get()
  2. Process frame through showDitheredPreview():
     a. Downsample camera frame (160x120) to OLED resolution (128x64)
     b. Average pixel values in each downsampled region
     c. Store results in gray_buffer array
     d. Normalize brightness values to full range
     e. Apply Floyd–Steinberg dithering algorithm
     f. Render dithered result to OLED display
  3. Return camera frame buffer using esp_camera_fb_return()
  4. Repeat continuously for live preview

DITHERING ALGORITHM (Floyd–Steinberg):
  For each pixel from top-left to bottom-right:
    1. Quantize current pixel (0 or 255)
    2. Calculate quantization error
    3. Distribute error to neighboring pixels:
       - Right: 7/16 of error
       - Bottom-left: 3/16 of error
       - Bottom: 5/16 of error
       - Bottom-right: 1/16 of error
        



          
Key Features:
          

            
Real-time camera frame capture at QQVGA resolution (160x120)
            
Grayscale image processing for efficient monochrome display
            
Floyd–Steinberg dithering algorithm for high-quality visual output
            
Automatic brightness normalization for optimal contrast
            
Continuous live preview stream to OLED display
            
Optimized downsampling from camera resolution to OLED resolution (128x64)
          
        


          
What You Need to Know:
          

            This code provides a foundation for displaying camera feeds on OLED displays. The Floyd–Steinberg dithering algorithm distributes quantization errors across neighboring pixels, creating smooth gradients and improved visual quality compared to simple thresholding. The code uses grayscale format for efficient processing, and the downsampling step averages multiple camera pixels into each OLED pixel to maintain image clarity.
          
          

            This implementation serves as a building block for the CircleSquareCamera AR system, where camera feeds are combined with game state overlays. See Ethical AI Use section for ChatGPT co-development transcript.
          
        


          

            
              
              View view_camera_oled.ino
            
          
          

            
              
              Download view_camera_oled.zip
            
          
          

            
              
              View camera_pins.h
            
          
          

            
              
              View ChatGPT Transcript
            
          
        
      


      

        
Group Assignment: CircleSquareCamera Augmented Reality System
        

          Complete Arduino code for the CircleSquareCamera AR system, including game board code that broadcasts player states via ESP-NOW, and camera board code that receives game packets and overlays player positions on live camera feeds.
        


        

          
Camera Board Code (CircleSquareCamera.ino)
          

            Camera board receives ESP-NOW packets from game boards, captures camera frames, processes them through dithering, and overlays game character positions (square for P1, circle for P2) on the live feed displayed on OLED.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, I²C, OLED display
  2. Initialize ESP-NOW in receive-only mode
  3. Register ESP-NOW receive callback
  4. Initialize camera (QQVGA, GRAYSCALE)
  5. Determine camera ID from MAC address (A or B)
  6. Display camera ready status

ESPNOW_RECEIVE_CALLBACK:
  1. Check if packet is correct size (NetPacket)
  2. Extract player state from packet
  3. Identify player (P1 or P2) based on sender MAC address
  4. Store player state in p1 or p2 variable
  5. Set gotP1 or gotP2 flag

LOOP:
  1. Capture camera frame (esp_camera_fb_get)
  2. Process frame through makeFrame():
     a. Downsample camera (160x120) to 64x32
     b. Normalize brightness
     c. Apply Floyd–Steinberg dithering
     d. Rotate 180° for correct orientation
  3. Draw camera feed to OLED (2x2 pixel blocks)
  4. Overlay game characters if received:
     - Draw square at p1.x, p1.y for Player 1
     - Draw circle at p2.x, p2.y for Player 2
  5. Display frame on OLED
  6. Return camera frame buffer
          



            
Key Features:
            

              
ESP-NOW receive-only mode for game state packets
              
MAC address-based player identification (P1: 0xAC, P2: 0x9C)
              
Real-time camera frame capture and dithering
              
180° rotation for correct display orientation
              
AR overlay of game characters on live camera feed
              
Automatic camera ID detection from MAC address
            
          


            

              
                
                View CircleSquareCamera.ino
              
            
            

              
                
                Download CircleSquareCamera.zip
              
            
            

              
                
                View camera_pins.h
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        


        

          
Game Board Code (TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino)
          

            Extended version of the CircleSquareShapeSongSwingalong game that broadcasts player state packets to both peer game boards and camera boards via ESP-NOW. This enables real-time AR visualization of gameplay on camera displays.
          


            
How the Code Works (Pseudocode):
            
SETUP:
  1. Initialize Serial, OLED display
  2. Set WiFi mode to STA
  3. Read self MAC address
  4. Initialize ESP-NOW
  5. Identify player role (P1 or P2) from MAC address
  6. Add peers: peer game board + Camera A + Camera B
  7. Register ESP-NOW send/receive callbacks
  8. Initialize touch sensors (warm-up readings)
  9. Reset player states (left/right positioning)

LOOP:
  1. Update touch sensor readings
  2. Process player input (left, right, jump)
  3. Update player physics (movement, gravity, collision)
  4. Create NetPacket with player state and ready flag
  5. Broadcast packet to all peers:
     - Other game board (peerMac)
     - Camera A (camA_MAC)
     - Camera B (camB_MAC)
  6. Render game state to OLED:
     - Draw ground line
     - Draw own character (square if P1, circle if P2)
     - Draw other player character
     - Display "Waiting Player" if other not ready
  7. Delay for frame timing (30ms)
          



            
Key Features:
            

              
ESP-NOW one-to-many broadcast to game boards and camera boards
              
MAC address-based player identification (P1: D8:3B:DA:75:05:AC, P2: D8:3B:DA:75:E1:9C)
              
Touch-based input controls with threshold detection
              
Real-time physics simulation (movement, gravity, collision)
              
Ready state synchronization for coordinated gameplay start
              
Game state rendering with square/circle character representation
            
          


            
What You Need to Know:
            

              This code extends the original CircleSquareShapeSongSwingalong game by adding camera board peers to the ESP-NOW network. The game broadcasts player state packets (position, velocity, onGround flag) to enable AR overlay on camera displays. The camera boards use these packets to draw game characters at the correct positions relative to the camera feed, creating an augmented reality experience. Player identification is done via MAC address comparison, with P1 using square representation and P2 using circle representation. See Ethical AI Use section for ChatGPT co-development transcript.
            
          


            

              
                
                View TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.ino
              
            
            

              
                
                Download TwoSquares_XiaoESP32S3_Touch_ReadySplash_MACFix_Camera.zip
              
            
            

              
                
                View ChatGPT Transcript
              
            
          
        
      
    


    

      
Reflections & Learnings
      
Key insights and learnings from working with networking and communications protocols.
    


    

      
Contributions
      
Acknowledgements and team roles for networking and communications work.
      

        

          ESP-NOW Multiplayer Game
          
Developed CircleSquareShapeSongSwingalong multiplayer game with ESP-NOW wireless communication during Week 2. See Week 2 documentation for details.
        
        

          Wi-Fi Camera Streaming System
          
Developed camera livestreaming and Edge AI face detection system during Week 10. See Week 10 documentation for details.
        
      
    


    

      
Ethical AI Use
      
Transparent documentation of AI assistance used in this week's networking and communications work.


        
AI-Assisted Camera Integration Game Update
        

          Used ChatGPT for collaborative development of the CircleSquareCamera augmented reality system, integrating ESP-NOW communication between game boards and camera boards. The AI assisted with ESP-NOW packet structure design, camera feed dithering algorithms, AR overlay implementation, and real-time synchronization of player states with camera displays. This extends the original CircleSquareShapeSongSwingalong multiplayer game from Week 2 with augmented reality visualization.
        
        

          
ChatGPT Co-Development Transcript:
          

            
              
              View ChatGPT Transcript
            
          
        
      


        
AI-Assisted Week 11 Page Creation and Assignment Documentation
        

          Cursor AI assisted with creating the complete Week 11 documentation page, including networking protocols overview, group assignment CircleSquareCamera AR documentation, individual assignment sections summarizing ESP-NOW multiplayer game and Wi-Fi camera streaming systems, network address tables, and all formatting and styling. The AI helped refine and organize content from Week 2 and Week 10 into professional, succinct networking documentation while preserving key technical details.
        
        

          The AI was used to:
        
        

          
Create week11.html from the week8.html template structure
          
Populate networking and communications resources from the MIT Academy class page
          
Summarize and integrate content from Week 2 (ESP-NOW multiplayer game) and Week 10 (Wi-Fi camera streaming)
          
Create network address tables with MAC and IP addresses for both networked systems
          
Format and style all sections for professional presentation
          
Embed images, videos, and design file download links
        
        

          
Complete AI Assistance Transcript:
          

            
              
              View HTML Transcript
            
            
              
              Download Markdown
            
          
        
      
    


      

        

          
            
          
          This work is licensed under a
          
            Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License