The early U.S. Territorial Period was 1821-1845, but the eventual Territorial Period lasted until 1912, when Arizona and New Mexico were admitted as states.

Where can you find territorial records?

I would suggest searching the online catalogs for these types of repositories, using the search term “territorial papers.”

1. State archives and libraries 
2. National Archives Regional Branches
3. The Family History Library in Salt Lake City.  Do a “keyword” search for “territorial papers” in the FamilySearch online catalog.
4. State level genealogical and historical societies
5. University Special Collections in the state you are researching
6. Territorial records can also be found on the county level sometimes

Here is a quick guide and links to the territorial papers available at the Family History Library:

• State Department territorial papers, Arizona, 1864-1872, FHL film 1580035
• State Department territorial papers: Colorado series, FHL film 1464017
• Territorial papers, Idaho, 1863-1872 FHL film 1580038
• Territorial papers of Montana, 1864-1872, FHL films 1602228 -9
• State department territorial papers, Nevada, 1861-1864 FHL film 1491200
• State department territorial papers: New Mexico, 1851-1972, FHL films 1580030-33
• State Department territorial papers, Utah series, FHL film 491567
• Interior Department territorial papers, Utah, 1850-1902, FHL films 1602234 -9
• Territorial papers of Wyoming, 1868-1873, FHL film 1602230

Here are some published finding aids for territorial records:

Kvasnicka, Robert M. The Trans-Mississippi West, 1804-1912: A Guide to Federal Records for the Territorial Period, pts. I-IV (Washington, District of Columbia : National Archives and Records Administration, c1993-1996).

Chiorazzi, Michael.  Pre-Statehood Legal Materials: A Fifty-State Research Guide, including New York City and the District of Columbia,  2 volumes (New York : The Haworth Information Press, 2005).

Some other good resources are:

United States, The public statutes at large of the United States of America / by authority of Congress (Boston : Little, Brown, n.d.)

United States. Congress. House and Senate Documents and Reports, United States Congressional Serial Set  (Washington : U.S. G.P.O., n.d.).

United States. Congress, American State Papers, 38 vol (Buffalo, N. Y. : W.S. Hein, 1998)