On stage, he was an introverted, shy, yet riveting solo performer. In his early days especially, he was often unkempt on stage and unconventional off it – a non-conformist who cared little for personal possessions and who often had no fixed address. It was as a member of the groundbreaking folk band Pentangle that he first achieved recognition beyond the folk scene. Formed in 1967, the band toured extensively until 1972, and although all the original members reunited in 1982, it was only Jansch and Jacqui McShee who stayed the course until the band folded in 1995.

His finger-picking playing style included a good deal of improvisation, bending the strings and varying the time signatures to fit the natural rhythm of the words of a song.

Jansch, whose forebears had come from Germany in the 19th century, was born in Glasgow but the family moved to Edinburgh, where he attended Ainslie Park secondary school. He worked, briefly, as a nurseryman, spending his early wages on a guitar. He sought lessons at the Howff folk club, wishing to emulate the guitar style of the American Big Bill Broonzy. Soon, Jansch had become resident unofficial caretaker at the Howff, spending much of his time developing his playing skills, with the Scottish singer Archie Fisher as a significant influence.

In the early 1960s, Jansch graduated from playing for his own pleasure to performing for an audience. He was one of the first guitarists to understand and then interpret and popularise Davy Graham’s guitar solo Anji. At the time, his personal, self-composed songs contrasted with the usual traditional or political repertoire of folk singers.

After busking in Europe in 1964, he moved to London, where his instrumental and songwriting skills were recognised by the producer Bill Leader, who recorded his eponymous first album, released on the Transatlantic label in 1965. The album included Needle of Death, a stark anti-drugs song written after a friend died of an overdose. His second record, It Don’t Bother Me, followed the same year.

It was a time of innovation in traditional song accompaniment. Graham had already brought his jazz and Arabic rhythms to a joint recording project with the folk singer Shirley Collins, and Jansch was by then greatly influenced by the young singer Anne Briggs. The traditional folk songs she taught him, plus his bluesy, improvised guitar accompaniment, dominated his third solo album, Jack Orion (1966), which featured John Renbourn on guitar. A joint album the same year, Bert and John, laid the foundations of Pentangle. Jack Orion included Blackwaterside, a traditional song Jansch learned from Briggs. His arrangement was copied by Page, who recorded the song with Led Zeppelin. The success of Jansch’s albums led to sell-out concerts in London and a tour of provincial city concert halls.

Renbourn was already performing with McShee when the idea of a band was suggested by Jansch; Danny Thompson (bass) and Terry Cox (percussion) added a jazz flavour to the mix of folk and blues. Pentangle’s performances were characterised by extended solos and improvisation, with McShee’s distinctive voice singing a mixture of traditional and band-composed songs. After a debut at the Royal Festival Hall in 1967, they began a relentless touring schedule as well as doing TV and radio work. The opening track of their third album, Basket of Light, was Light Flight, which became the theme tune of the popular television series Take Three Girls (1969). The album reached number five in the charts, and the band appeared on Top of the Pops.

During the Pentangle years, Jansch recorded three solo albums, notably Rosemary Lane (1971), a stark, reflective work that included the traditional song Reynardine, learned from Briggs, alongside his own compositions.

Jansch found the touring with Pentangle too much, and he forced the band to split in early 1973. He retreated to his farm in Wales, but he needed musical challenges, and also to relearn his craft as a solo performer. By the time his album LA Turnaround was released in 1974, he had separated from his second wife, Heather, and moved back to London. At this point, his heavy drinking was taking its toll on his performances and reliability.

Appreciative audiences worldwide and the need to earn a living meant a return to international touring, and Jansch teamed up with the multi-instrumentalist Martin Jenkins. Their concept album Avocet (1979) contained an 18-minute title track inspired by the traditional song the Cuckoo, and five further pieces named after birds.

Pentangle re-formed in 1982, but within a couple of years Renbourn, Thompson and Cox had left. Replacements were found, but the nostalgia surrounding the part-time band had a detrimental effect on Jansch’s already diminishing solo career.

In 1987 Jansch became seriously ill and he gave up alcohol. His biographer, Colin Harper, wrote that “Bert’s creativity, reliability, energy, commitment and quality of performance were all rescued dramatically” by this decision. In 1995 he left Pentangle, which then re-formed as Jacqui McShee’s Pentangle. His back catalogue emerged on CD, and a new generation of musicians discovered his work.

He continued to write songs and make albums. When the Circus Comes to Town (1995), with its tribute to the doctor who saved him – The Lady Doctor from Ashington – led to extensive international touring and a television appearance on Later with Jools Holland. The television documentaries Acoustic Routes (1992) and Dreamweaver (2000), as well as Harper’s biography, Dazzling Stranger (2000), helped cement his renewed reputation.

In 2001 Jansch received a lifetime achievement award at the BBC Radio 2 Folk Awards and, in 2007, so did Pentangle. The original line-up performed at the award ceremony and on a 2008 reunion tour. He received an honorary doctorate from Edinburgh Napier University in 2007. That year, he performed with the Babyshambles singer Pete Doherty, and the singer-songwriter Beth Orton guested on his 2006 album The Black Swan.

Jansch had heart surgery in 2005, and a further operation for lung cancer led to the cancellation of his 2009 tour of the US. But in the summer of 2010, he joined Young on his Twisted Road tour of the US. Last summer, he and the other original members of Pentangle were reunited at Glastonbury, Cambridge folk festival and the Royal Festival Hall.

Jansch was married three times: briefly to Lynda Campbell in 1963, to the sculptor Heather Jansch, and to Loren Auerbach, who survives him. He is also survived by his sons, Kieron and Adam. Another son, Richard, predeceased him.

Robin Denselow writes: Bert Jansch was that rarity, a musician who really did deserve to be regarded as a legend, and who retained that status throughout his career. He was an extraordinary guitarist and a thoughtful songwriter, and generations of would-be pickers sat at his concerts watching his fingerwork with envy and astonishment.

He was influenced by traditional songs, blues and the “folk-baroque” of Davy Graham, but his distinctive style always allowed him to take chances and work with different musicians. When I first met him, as a student journalist in the 1960s, he was outselling Bob Dylan in the folk shops along the Charing Cross Road, and told me: “I’m not recording for anyone, just myself.”

Years later, visiting him at his garden flat in Kilburn, it always struck me how little he had changed – he was still a tousled-haired figure with a slight mumble and quiet sense of humour, happiest when picking up a guitar and discussing music. One of the most memorable of Bert’s shows was his 60th birthday celebration at London’s South Bank, when he ran through the full gamut of his work, joined by younger fans including Johnny Marr, Bernard Butler and Hope Sandoval. He was a unique performer.

• Bert (Herbert) Jansch, guitarist, born 3 November 1943; died 5 October 2011