Purpose Numerous reports indicate that tumor-infiltrating lymphocytes (TILs) are a prognostic factor in various cancers and that they must be good biomarkers. However, the methods of evaluating TILs differ in each study; thus, there is not yet a standardized methodology for evaluating TILs. The purpose of this study is to evaluate the prognostic significance of tumor-infiltrating lymphocytes (TILs) in patients with colorectal cancer (CRC) using the new method proposed by the International TILs Working Group in breast cancer and to standardize the method of evaluating TILs in CRC. Methods We retrospectively reviewed a database of 160 patients with Stage II or III CRC. The density of TILs was assessed by measuring the area occupied by mononuclear cells over the stromal area on hematoxylin and eosin (H-E)-stained sections. We set 42% as the cut-off percentage of the area occupied by TILs according to the receiver operating characteristic curve, and we classified patients into the high-TILs and the low-TILs groups. Results The rates of relapse-free survival (RFS) and overall survival (OS) in the high-TILs group were significantly higher than those in the low-TILs group. A multivariate analysis showed that the density of TILs was independently associated with RFS and OS. Moreover, the density of TILs assessed by an observer was significantly associated with the density of TILs assessed by the automated imaging software program. Conclusions The new method for evaluating TILs, which was recommended by the International TILs Working Group in breast cancer, might be a useful predictive factor in colorectal cancer patients.