IA Tools Plugin

A comprehensive FastPluggy plugin for AI/ML infrastructure monitoring and management.
This plugin provides essential tools for monitoring GPU resources, CUDA environments, and Hugging Face model caches.

Overview

The IA Tools plugin is designed to help developers and system administrators monitor and manage AI/ML infrastructure components. It provides web-based interfaces for checking GPU status, CUDA availability, and managing Hugging Face model caches.

Features

🖥️ GPU Information & Monitoring

• Real-time GPU metrics: Memory usage, utilization, temperature, power consumption, and fan speed
• GPU process monitoring: Track processes running on GPUs with detailed system information
• Multi-GPU support: Monitor multiple NVIDIA GPUs simultaneously
• Process details: View PID, memory usage, username, process name, and command line for GPU processes

🔧 CUDA Status & Environment Checking

• CUDA availability detection: Check if CUDA toolkit is installed and accessible
• CUDA version reporting: Display installed CUDA version information
• PyTorch GPU support: Verify PyTorch installation and GPU compatibility
• Environment validation: Comprehensive status dashboard for AI/ML development environment

🤗 Hugging Face Cache Management

• Cache directory detection: Automatically locate Hugging Face model cache
• Model inventory: List all cached models with detailed information
• Storage analysis: Display cache size, file counts, and storage usage
• Model type detection: Identify PyTorch, TensorFlow, ONNX, and Transformers models
• Environment variable support: Supports HF_HOME and HUGGINGFACE_HUB_CACHE configuration

Installation

Prerequisites

• Python >= 3.10
• FastPluggy framework
• NVIDIA GPU (optional, for GPU monitoring features)
• CUDA toolkit (optional, for CUDA status features)

Dependencies

The plugin automatically installs the following dependencies:
- pynvml: NVIDIA GPU monitoring library
- psutil: System and process monitoring
- nvidia-ml-py3: Alternative NVIDIA ML library

Install from Source

cd src/fastpluggy_plugin/ia_tools
pip install -e .

Configuration

Environment Variables

Hugging Face Cache Configuration

• HUGGINGFACE_HUB_CACHE: Custom cache directory path (highest priority)
• HF_HOME: Hugging Face home directory (cache will be $HF_HOME/hub)
• Default: ~/.cache/huggingface/hub

System Configuration

• PROCFS_PATH: Custom /proc filesystem path for containerized environments

Plugin Configuration

The plugin is automatically registered with FastPluggy and appears in the "IA" menu section.

Usage

Web Interface

Once installed, the plugin provides three main pages accessible through the FastPluggy web interface:

1. GPU Info (/gpu_info)

• View detailed information about all available NVIDIA GPUs
• Monitor real-time GPU metrics and utilization
• Track GPU processes and their resource usage

2. CUDA Status (/cuda_status)

• Check CUDA toolkit availability and version
• Verify PyTorch GPU support and configuration
• View comprehensive environment status dashboard

3. Hugging Face Cache (/huggingface_cache)

• Browse cached Hugging Face models
• View cache summary and storage statistics
• Analyze model types and sizes

Programmatic Access

GPU Monitoring

from fastpluggy_plugin.ia_tools.tools.nvidia_gpu_tool import get_gpu_info, get_gpu_processes

# Get GPU information
gpu_info = get_gpu_info()
for gpu in gpu_info:
    print(f"GPU {gpu['gpu_index']}: {gpu['name']}")
    print(f"Memory: {gpu['used_memory']}/{gpu['total_memory']} MiB")
    print(f"Utilization: {gpu['utilization']}%")

# Get GPU processes
processes = get_gpu_processes()
for proc in processes:
    print(f"PID {proc['pid']}: {proc['process_name']} using {proc['memory_usage']} MiB")

Environment Checking

from fastpluggy_plugin.ia_tools.widgets import check_nvidia_gpu, check_cuda_version, check_python_gpu_support

# Check NVIDIA GPU availability
has_gpu, gpus = check_nvidia_gpu()
if has_gpu:
    print(f"Found {len(gpus)} GPU(s)")

# Check CUDA version
has_cuda, version = check_cuda_version()
if has_cuda:
    print(f"CUDA version: {version}")

# Check PyTorch GPU support
has_torch, info = check_python_gpu_support()
if has_torch and info['cuda_available']:
    print(f"PyTorch {info['version']} with {info['gpu_count']} GPU(s)")

Hugging Face Cache Management

from fastpluggy_plugin.ia_tools.tools.huggingface_cache_tool import get_cached_models, get_cache_summary

# Get cache summary
summary = get_cache_summary()
print(f"Cache directory: {summary['cache_directory']}")
print(f"Total models: {summary['total_models']}")
print(f"Total size: {summary['total_size_human']}")

# List cached models
models = get_cached_models()
for model in models:
    print(f"{model['full_name']}: {model['size_human']} ({model['model_type']})")

API Endpoints

All endpoints require authentication and return HTML responses for web interface display.

Error Handling

The plugin includes comprehensive error handling:
- Graceful degradation when NVIDIA drivers are not available
- Safe handling of missing CUDA installations
- Robust cache directory detection with fallbacks
- Process monitoring with permission handling

Troubleshooting

GPU Monitoring Issues

• Ensure NVIDIA drivers are installed and up-to-date
• Verify nvidia-smi command is available in PATH
• Check that the user has permissions to access GPU information

CUDA Detection Problems

• Install CUDA toolkit if not present
• Ensure nvcc is available in PATH
• Verify CUDA installation with nvcc --version

Hugging Face Cache Issues

• Check cache directory permissions
• Verify environment variables are set correctly
• Ensure sufficient disk space for cache operations

Development

Project Structure

src/fastpluggy_plugin/ia_tools/
├── src/
│   ├── plugin.py              # Main plugin class
│   ├── router.py              # FastAPI routes and endpoints
│   ├── widgets.py             # Environment checking utilities
│   └── tools/
│       ├── nvidia_gpu_tool.py # GPU monitoring functions
│       └── huggingface_cache_tool.py # HF cache management
├── pyproject.toml             # Package configuration
├── requirements.txt           # Additional dependencies
└── README.md                  # This file

Contributing

1. Follow the existing code style and patterns
2. Add appropriate error handling and logging
3. Update documentation for new features
4. Test with various GPU configurations and environments

License

This plugin is part of the FastPluggy ecosystem. Please refer to the main FastPluggy project for license information.

Version History

• 0.0.7: Current version with full GPU monitoring, CUDA status, and Hugging Face cache management
• 0.0.5: Added Hugging Face cache functionality
• Earlier versions: Basic GPU monitoring capabilities

Support

For issues, feature requests, or contributions, please refer to the main FastPluggy project repository.