File

The File Loader is a versatile document loader that supports multiple file formats including TXT, JSON, CSV, DOCX, PDF, Excel, PowerPoint, and more. This module provides a unified interface for loading and processing various file types.

This module provides a sophisticated file loader that can:

• Load multiple file formats
• Support both base64-encoded files and files from storage
• Handle PDF-specific processing options
• Process JSON and JSONL with pointer extraction
• Support text splitting
• Customize metadata extraction
• Handle file storage integration

Inputs

Required Parameters

• File: The file(s) to process (supports multiple formats)

Optional Parameters

• Text Splitter: A text splitter to process the extracted content
• PDF Usage: Choose between:
  • One document per page
  • One document per file
• Use Legacy Build: Use legacy build for PDF compatibility issues
• JSONL Pointer Extraction: Pointer name for JSONL files
• Additional Metadata: JSON object with additional metadata
• Omit Metadata Keys: Comma-separated list of metadata keys to omit

Outputs

• Document: Array of document objects containing metadata and pageContent
• Text: Concatenated string from pageContent of documents

Supported File Types

• Text Files (.txt)
• JSON Files (.json)
• JSONL Files (.jsonl)
• CSV Files (.csv)
• PDF Files (.pdf)
• Word Documents (.docx)
• Excel Files (.xlsx, .xls)
• PowerPoint Files (.pptx, .ppt)
• And more…

Features

• Multi-format support
• Storage integration
• PDF processing options
• JSON pointer extraction
• Text splitting support
• Metadata customization
• Error handling
• MIME type detection

File Processing Options

PDF Processing

• Per-page splitting
• Single document mode
• Legacy build support
• OCR compatibility

JSON/JSONL Processing

• Pointer-based extraction
• Structured data handling
• Array processing
• Nested object support

Notes

• Automatically detects file type
• Handles multiple files simultaneously
• Supports file storage integration
• Preserves file metadata
• Handles large files efficiently
• Error handling for invalid files
• Memory-efficient processing