Simply, the answer is because of the long polymer chains.
The band is made of many, many, many molecules that are chain like. Actually we can work out how many! Lets say an elastic band weighs about 0.25 grams and the molar mass of cellulose (a natural polymer) is around 162g. This means there are 0.0015432 moles of cellulose in my elastic band. Multiply that by Avogadro’s constant and you get around 9×10^20 molecules. This is the same as 900 billion billion… When you put a force on the band these chains can realign in order to straighten up. When released the chains want to return to a more chaotic state (ENTROPY). This process also releases heat, this is why you can stretch and elastic band lots, put it to your lips and feel heat.
How can we use this information to understand how different polymer things stretch? Materials with longer chains attract each other more and are therefore more resistant to stretch. Cross linking can also be a factor, this is where there are bonds between chains. These cross bonds will reduce potential for chains to shift. This can be found in vulcanised rubber. Sulphur, among other things, is introduced to natural rubber (made of cellulose). Interestingly this process is named after Vulcan, the roman god of fire.