The curiously named !!!, whose moniker can be pronounced by repeating any one-syllable percussive sound three times (e.g., chk chk chk), formed in 1996 after the demise of the mid-'90s hardcore act Yah Mos. While on tour, members of Yah Mos, including !!! singer Nic Offer, envisioned forming a band oriented more toward danceable music, and once they returned to Sacramento, CA, their hometown, they decided to turn their working concept into an actual group.

The band quickly grew into an eight-member ensemble, including Mario Andreoni on guitar, Offer on vocals, Justin van der Volgen on bass, Dan Gorman on trumpet and percussion, Tyler Pope on guitar, Allan Wilson on saxophone and percussion, John Pugh on drums, and Jason Racine on percussion. Fusing the groove-oriented style of James Brown's backing band with the edginess of Gang of Four and the Contortions and featuring an expansive percussive presence, !!! spent its first few years as a group developing its sound and playing at house parties for friends in Sacramento. At the same time, Offer, Pope, and van der Volgen were playing in a dub-inflected instrumental outfit called Out Hud with Molly Schnick and Phyllis Forbes, two musicians who used to be in the band Raoul.

After putting out a 7 on Hopscotch Records and a split single with Outhud in 1998, the two bands decided to release a split full-length together in 1999, a release jointly sponsored by the label Gold Standards Laboratories, founded by Sonny Kaye of Angel Hair and the VSS fame, and Zum, a San Francisco-based zine. After extensive touring of the States, which gave them a chance to further expand the songs they had written by placing them in the interactive context of a live show, !!! recorded and released a self-titled album on GSL in December 2000. Soon after, the band re-entered the studio and recorded enough material for a second album. Members of the band moved to New York, but the group still remained intact, planning regional tours and tape trades across the expanses of the Midwest. In 2003, the band issued Me and Giuliani Down by the School Yard (A True Story), their first release for Touch & Go.