Older individuals exhibit a diminished ability to respond to and clear respiratory pathogens and, as such, experience a higher rate of lung infections with a higher mortality rate. It is unclear why respiratory pathogens impact older people disproportionately. Using human lung tissue from donors aged 22-68 years, we assessed how the immune cell landscape in lungs changes throughout life and investigated how these immune cells respond following in vitro exposure to influenza virus and SARS-CoV-2. While the frequency of most immune cell subsets profiled in the human lung remained stable with age, memory CD8+ T cells declined, with the tissue-resident memory (Trm) CD8+ T cell subset being most susceptible to age-associated attrition. Infection of lung tissue with influenza virus resulted in an age-associated attenuation in the antiviral immune response. In contrast, irrespective of donor age, exposure of human lung cells to SARS-CoV-2, did not trigger local immune cell activation nor did it trigger a proinflammatory response. Our findings show that the attrition of tissue-bound pathogen-specific Trm in the lung that occurs with advanced age results in a diminished early antiviral immune response which creates a window of opportunity for respiratory pathogens to gain a greater foothold.