Per-map benchmark results across 5 algorithms, 4 maps, 8–128 agents. All data from simulation experiments (20 instances per configuration).
| Learning-based | Hybrid | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SCRIMP | DCC | MAPF-GPT-2M | EPH | HI-MAPF | |||||||||||
| SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | |
| 10 | 100 | 48.0 | 72.8× | 100 | 49.1 | 7.3× | 100 | 49.2 | 142× | 100 | 47.6 | 5.5× | 100 | 50.5 | 3.6 (1×) |
| 20 | 100 | 50.5 | 30× | 100 | 58.0 | 11× | 100 | 53.8 | 137× | 95 | 62.5 | 11× | 100 | 56.0 | 18.8 (1×) |
| 30 | 100 | 52.4 | 13× | 100 | 59.5 | 10.5× | 100 | 53.9 | 82× | 100 | 62.0 | 8.7× | 100 | 55.5 | 66.3 (1×) |
| 40 | 95 | 58.0 | 28.2× | 95 | 72.8 | 28.8× | 100 | 57.2 | 205× | 100 | 64.5 | 28.7× | 100 | 107.0 | 48.8 (1×) |
| 50 | 100 | 56.8 | 7.4× | 90 | 81.7 | 15× | 100 | 58.9 | 76× | 95 | 79.0 | 14× | 100 | 61.4 | 221.4 (1×) |
| 60 | 90 | 71.4 | 13× | 70 | 92.0 | 30× | 100 | 63.0 | 120× | 95 | 88.7 | 29× | 100 | 128.2 | 214.2 (1×) |
| 70 | 90 | 70.5 | 1.5× | 60 | 104.5 | 4.8× | 100 | 68.2 | 17× | 65 | 110.7 | 5.4× | 100 | 65.5 | 2,158.2 (1×) |
| 80 | 80 | 83.8 | 3.3× | 60 | 111.7 | 12× | 100 | 72.9 | 37× | 70 | 108.9 | 14× | 100 | 71.8 | 1,405.5 (1×) |
| 90 | 60 | 96.9 | 3.2× | 30 | 117.8 | 15× | 100 | 70.0 | 34× | 25 | 123.8 | 18× | 95 | 82.8 | 1,950.7 (1×) |
| 100 | 60 | 97.2 | 1.9× | 40 | 117.5 | 10× | 100 | 77.0 | 23× | 50 | 121.0 | 11× | 85 | 76.5 | 3,693.9 (1×) |
| 128 | 70 | 96.6 | – | 0 | – | – | 90 | 94.1 | – | 5 | 127.5 | – | 0 | – | – |
| Learning-based | Hybrid | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SCRIMP | DCC | MAPF-GPT-2M | EPH | HI-MAPF | |||||||||||
| SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | |
| 10 | 50 | 112.5 | 680× | 100 | 97.7 | 12× | 100 | 99.6 | 350× | 95 | 97.3 | 8.7× | 100 | 108.0 | 0.9 (1×) |
| 20 | 10 | 126.5 | 223× | 100 | 107.8 | 9.6× | 100 | 110.2 | 231× | 100 | 105.7 | 7.7× | 100 | 115.6 | 6.2 (1×) |
| 30 | 10 | 126.7 | 94× | 90 | 112.2 | 7.4× | 100 | 111.7 | 147× | 100 | 109.5 | 6.1× | 100 | 118.2 | 23.4 (1×) |
| 40 | 0 | – | – | 100 | 112.8 | 8.5× | 100 | 112.7 | 137× | 100 | 112.3 | 7.3× | 100 | 122.6 | 45.0 (1×) |
| 50 | 0 | – | – | 85 | 116.8 | 8.1× | 100 | 115.2 | 90× | 100 | 113.8 | 5.8× | 100 | 124.0 | 109.8 (1×) |
| 60 | 0 | – | – | 65 | 120.8 | 11× | 85 | 117.7 | 130× | 95 | 116.1 | 8.7× | 100 | 130.2 | 110.8 (1×) |
| 70 | 0 | – | – | 50 | 123.5 | 10× | 95 | 118.7 | 112× | 85 | 120.8 | 9.0× | 100 | 128.2 | 183.0 (1×) |
| 80 | 0 | – | – | 20 | 126.8 | 14× | 95 | 121.0 | 125× | 90 | 120.2 | 12× | 100 | 128.7 | 211.9 (1×) |
| 90 | 0 | – | – | 60 | 124.6 | 17× | 75 | 121.3 | 140× | 80 | 122.2 | 13× | 95 | 130.4 | 243.6 (1×) |
| 100 | 0 | – | – | 25 | 127.2 | 13× | 75 | 123.8 | 89× | 45 | 125.3 | 11× | 90 | 130.1 | 472.9 (1×) |
| 128 | 0 | – | – | 0 | – | – | 55 | 126.2 | 132× | 15 | 127.7 | 23× | 5 | 132.0 | 530.0 (1×) |
| Learning-based | Hybrid | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SCRIMP | DCC | MAPF-GPT-2M | EPH | HI-MAPF | |||||||||||
| SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | |
| 8 | 20 | 226.2 | 321× | 100 | 114.2 | 6.3× | 100 | 114.0 | 83× | 100 | 111.9 | 3.6× | 100 | 115.3 | 3.6 (1×) |
| 16 | 0 | – | – | 100 | 130.7 | 8.8× | 100 | 120.9 | 67× | 100 | 122.2 | 5.9× | 100 | 124.0 | 21.8 (1×) |
| 32 | 0 | – | – | 85 | 163.5 | 11× | 100 | 126.4 | 61× | 100 | 127.0 | 6.0× | 100 | 136.7 | 105.2 (1×) |
| 64 | 0 | – | – | 90 | 190.7 | 11× | 100 | 137.8 | 36× | 100 | 146.1 | 8.8× | 100 | 153.1 | 759.9 (1×) |
| 128 | 0 | – | – | 5 | 254.8 | – | 35 | 234.0 | – | 90 | 207.7 | – | 0 | – | – |
| Learning-based | Hybrid | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SCRIMP | DCC | MAPF-GPT-2M | EPH | HI-MAPF | |||||||||||
| SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | SR↑ | MS↓ | IU↓ | |
| 8 | 0 | – | – | 100 | 191.8 | 5.4× | 100 | 193.6 | 134× | 100 | 191.6 | 2.1× | 100 | 192.8 | 0.8 (1×) |
| 16 | 0 | – | – | 100 | 213.3 | 16× | 100 | 214.0 | 510× | 100 | 212.9 | 11× | 100 | 215.1 | 1.0 (1×) |
| 32 | 0 | – | – | 100 | 219.2 | 8.3× | 100 | 221.8 | 215× | 100 | 218.1 | 5.4× | 100 | 220.3 | 10.7 (1×) |
| 64 | 0 | – | – | 100 | 227.6 | 15× | 100 | 229.3 | 401× | 100 | 227.8 | 11× | 100 | 227.6 | 23.6 (1×) |
| 128 | 0 | – | – | 95 | 236.8 | 14.5× | 95 | 237.2 | 277× | 90 | 239.1 | 10.8× | 100 | 234.7 | 139.1 (1×) |
5 TurtleBot4 robots in a 6×6 physical grid, ROS 2, distributed deployment. Results averaged across 5 problem instances.
| Metric | SCRIMP | DCC | MAPF-GPT | EPH | HI-MAPF |
|---|---|---|---|---|---|
| Success Rate (%) | 100 | 100 | 100 | 100 | 100 |
| Avg. Makespan | 16.4 | 19.4 | 13.6 | 21.8 | 12.2 |
| Avg. Total IU | 42,413 | 596 | 588 | 623 | 199 |
| IU vs HI-MAPF | 213× | 3.0× | 3.0× | 3.1× | 1× |
| Avg. Comm. Freq. | 180 | 116 | 82 | 131 | 10 |
| Avg. Exec Time (s) | 211.2 | 235.0 | 176.2 | 234.9 | 181.5 |
| Avg. Exec Msgs | 169 | 199 | 141 | 223 | 127 |
Individual results for each of the 5 problem instances on a 6×6 grid with 5 TurtleBot4 robots.
| Learning-based | Hybrid | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SCRIMP | DCC | MAPF-GPT-2M | EPH | HI-MAPF | |||||||||||
| MS | IU | Exec(s) | MS | IU | Exec(s) | MS | IU | Exec(s) | MS | IU | Exec(s) | MS | IU | Exec(s) | |
| Prob 1 | 14 | 37,128 | 185.4 | 18 | 540 | 214.8 | 12 | 504 | 162.3 | 20 | 560 | 218.7 | 11 | 180 | 168.2 |
| Prob 2 | 18 | 45,720 | 225.6 | 20 | 620 | 241.5 | 14 | 612 | 182.4 | 22 | 648 | 240.1 | 13 | 210 | 190.5 |
| Prob 3 | 15 | 39,840 | 198.2 | 19 | 580 | 228.6 | 13 | 556 | 170.8 | 21 | 602 | 228.4 | 12 | 195 | 175.3 |
| Prob 4 | 17 | 44,256 | 218.9 | 21 | 640 | 248.2 | 15 | 624 | 185.6 | 23 | 670 | 245.8 | 13 | 208 | 188.7 |
| Prob 5 | 18 | 45,121 | 228.1 | 19 | 600 | 241.8 | 14 | 644 | 180.0 | 23 | 635 | 241.5 | 12 | 202 | 184.6 |