ko te robots.txt he kōnae tuhinga e whakaputahia ana e ngā paetukutuku i te taumata teitei o ia kaiwhakarato (hei tauira, example.com/robots.txt) e tohu ana i ngā whārangi, i ngā wāhanga rānei e taea ana e ngā pūtirotiro miihini rapu me ētahi atu bot te uru ki, te mea rānei kāore e taea. He paerewa voluntary tēnei: ka whakaute ngā pūtirotiro e pai ana, engari kāore he tikanga hangarau e taea ana e ia.

E ai ki te RFC 9309, me whai i ngā ture kei roto i te kōnae robots.txt i tangohia e ngā pūtirotiro, engari e kī ana hoki kei te kore ēnei ture he momo whakaaetanga uru.

Ka taea e te kōnae robots.txt te whakatakoto i ngā ture rerekē mō ngā pūtirotiro rerekē i runga i te ingoa, nō reira ka taea e te paetukutuku te tuku i te uru whānui ki te pūtirotiro miihini rapu kotahi, i te mea e aukati ana i ētahi atu.

Ngā Kupu Pānga