IEEE

QMLFQ: A Quantization-based Queue Scheduling Framework for Efficient LLM Serving

Abstract: Fueled by advancements in Large Language Models (LLMs), applications like ChatGPT have ushered in a transformative era for AI-powered interactions. However, the computational demands of LLMs ...
IEEE

PF2A-ViT: Parameter-Free and Feature-Aware Dynamic Token Pruning Accelerator With Complementary Quantization-Encoding for Vision Transformer

Abstract: Vision Transformers (ViTs) have achieved outstanding performance in visual applications. However, ViT’s increasing parameters and computation overhead limit its deployment on hardware.