Vaccine providers should be reporting to the state health department, according to my public health department nurse daughter. What gets reported and where it might go from there depends on the states.
My wife and I got our shots at a drive-through mass vaccination clinic run by Oregon Health Sciences University (OHSU). My vaccination showed up on my OHSU "MyChart" page. I snapped a picture of my CDC card and sent it in a message to my PCP, who is part of The Portland Clinic (TPC, not affiliated with OHSU). My PCP nurse replied she had added to vaccination to my record at TPC. After the second shot I just messaged my PCP I had gotten the second shot. Both jabs show on my MyChart vaccination page.
My wife, however, did not had a medical record at OHSU. A couple weeks ago, however, she checked her MyChart account with her PCP (with Providence health, also unaffiliated with OHSU) and found her vaccinations had made their way to her PCP records without any action by her.
Generally: If you have online access to your health care provider (e.g. MyChart or equivalent), check to see if your vaccination shows on your records. If not, I would message your PCP, notify them you are vaccinated, and ask that they verify with the state and add it to your records. Then there should be an official medical record that would carry more weight if the CDC paper cards become untrustworthy.