The Secret Man Behind the World’s Most Visible Building
By Jason M. Barr and Ann Berman
The Empire State Building (ESB) is arguably the most iconic building of the modern era. Its presence identifies New York City on screens large and small. King Kong clung to its spire; in 1945, the pilot of a B-25 Bomber, lost in a fog, crashed into its 78th floor; and during the Roaring Twenties, it emerged as the victor in the race to claim the title of “world’s tallest skyscraper.”
The Empire State Building. Photo by Jason Barr.
And yet almost all the stories about the origins of this New York landmark, online and in print, are inaccurate. They all omit the pivotal, behind-the-scenes role played by Louis Graveraet Kaufman (LGK) (1870-1942), the secret schemer, without whom the ESB would not have been built. LGK’s hidden machinations irrevocably changed Gotham — and world — history, yet few today know his name.
Louis Graveraet Kaufman
Louis Graveraet Kaufman. Photo is courtesy of Peter Kaufman, Marquette Michigan.
It is a name that was once familiar in New York City. The son of a German Jewish peddler of men’s furnishings and a part-Native-American mother, Kaufman grew up in Marquette, Michigan, on the south shore of Lake Superior. After high school, he took a series of undistinguished jobs and seemed destined for mediocrity until 1899, when he went to work for the local savings bank. Almost overnight, Kaufman revealed himself to be a financial savant able to grasp complicated financial and managerial concepts and utilize them with charm and finesse — techniques he would later apply to help make the Empire State Building a reality.
During the next few years, he married Marie Young, the daughter of a Chicago millionaire; took over the National Bank of Marquette; became president of the Michigan Bankers Association; and joined the Executive Committee of the American Bankers Association. His precipitous rise did not go unnoticed. In 1910, Elbert Gary, the powerful head of U. S. Steel, sent Kaufman to New York City with a mandate to combine two underperforming banks into a successful new one. Kaufman quickly grew the Chatham Phenix National Bank into a Wall Street powerhouse and then, through clever financial sleight of hand, made it the only national bank in the city with a system of branches.
But the Kaufman juggernaut didn’t end with banking. In 1915, he financed General Motors founder William Durant’s bid to reclaim control of his company, then changed its destiny. First, the two men collaborated on a series of complicated maneuvers to put the majority of GM shares back in Durant’s hands. Then, on the day of a pivotal board meeting, with control of the company still uncertain, Kaufman invited Chatham Phenix client, Pierre S. Du Pont of the mighty Delaware gunpowder company, to be present. Quite unexpectedly, Du Pont was invited to break the power stalemate by naming three new members to an enlarged GM board. He brought Du Pont stalwarts on board, including his financial guru, John J. Raskob. Although the move temporarily returned Durant to power, thanks to LGK, the Du Ponts were now in the mix and would run General Motors from 1920 into the 1950s.
High Society
Throughout the 1920s, Kaufman’s wealth continued to grow, and he and his family lived a life of luxury and social advancement. In the summer, celebrity guests like Mary Pickford, Ethel Barrymore, and Fred Astaire traveled by private train car to the Kaufmans’ 26,000 square-foot lodge on the shore of Lake Superior, to attend parties that sometimes went on for days. The Kaufman daughters were feted at debutante balls and presented at the Court of St. James, and the sons married Broadway dancers. In the wintertime, the Kaufmans removed to Palm Beach, where they entertained lavishly at their capacious villa and aboard their houseboat moored in Lake Worth.
Newspapers across the country chronicled the family’s every move. Marie Kaufman had a stable of racehorses, a famous collection of jewels, and soon, a fabulous new apartment. In 1929, likely because no one named “Kaufman” was welcome to rent on Park Avenue, Louis Kaufman built his own apartment building. He bought a lot at 625 Park Avenue and commissioned an elegant, neo-classical design from architect James Edwin Ruthven Carpenter. The triplex penthouse occupied by the Kaufmans, now owned by Henry Kravis, remains one of the most famous pre-war apartments in the city.
The Birth of an Icon
That same year, Kaufman received a visit from John J. Raskob. Raskob had served on the board of the Chatham Phenix, and Kaufman had been a board member at GM when Raskob was running the show, so the two men knew each other well. Now Raskob knew something else. Thanks to a creditor’s failure to make a loan payment, LGK was in the control of a prime building spot at Fifth Avenue and 34th Street. Raskob had an idea of how to use that land, and it was a big one. He and Kaufman were going to build the tallest building in the world.
Raskob had good reason to embark on such a project. He had been fired from GM for backing former New York governor Al Smith for president against Herbert Hoover in 1928 (GM president Alfred P. Sloan was a staunch Hoover man) and rebuffed by Walter P. Chrysler, whose company he’d hoped to run instead. He needed a project, particularly one that would top the skyscraper Chrysler was building and employ his friend Al Smith, who was out of work and hurting for money. And he needed Kaufman—a banker with a large pot of money at his fingertips and control of a huge building site with no height restrictions.
For Louis Kaufman, it was different. The twenties were roaring, and his bank was soaring. He needed nothing, yet he was attracted to Manhattan’s most exciting and perilous game — real estate. Kaufman had seen the fortune others were making in the booming commercial real estate market, and he relished the chance to become the progenitor of an iconic building that would stand the test of time.
The Chain of Events
625 Fifth Avenue. Apartment Building Constructed by Louis Graveraet Kaufman. Photo by Ann Berman.
Since 1893, the corner of Fifth Avenue and 34th Street had been home to one of New York’s most famous hotels — the Waldorf-Astoria. Over the decades, its ballrooms hosted lavish parties, and its suites housed Gilded Age elites. But by the 1920s, the hotel was a relic. The ponderous building was expensive to maintain, and New York’s society was increasingly living — and socializing — further uptown. In December 1928 the hotel’s owners announced that they were selling the site.
The winning bidder was Bethlehem Engineering, a company founded and run by Floyd deL. Brown. He was the consummate New York real estate man who knew the field as well as anyone. He was born in New York City in 1885, studied architecture in Paris, and received an engineering degree from Columbia University. In 1918, he organized the Bethlehem Engineering Company to engage in real estate development and by 1928, he had a long track record of successful projects.
Brown planned to build a 50-story loft, office, and showroom skyscraper, encompassing two million square feet. His company negotiated the deal with an initial burst of financing from the Chatham Phenix National Bank, and then, in February 1929, secured a $24 million construction loan from the Metropolitan Life Insurance Company. However, the MetLife loan, as large as it was, was not sufficient to cover the total cost. Brown needed to find another $10 – 12 million to make it all work. In the end, he could not raise the money, and he failed to make his scheduled payment to the bank. Chatham Phenix took over his claim.
Typically, a bank would not want to get involved in real estate development and would quickly get the contract off its books. But here is where Kaufman’s actions became pivotal, as he directly inserted himself into the lot's future. If he had decided to let the whole thing go, who knows what would have happened to the site, but instead, he was that rare banker excited to try his hand at Manhattan real estate.
Kaufman’s Machinations
After Brown defaulted, Kaufman quickly formed a syndicate of like-minded men to proceed where Brown left off. The members pooled their equity, took out a loan from Chatham Phenix Bank for the remainder, and bought the lot from the bank. But after Raskob’s fateful visit, the property changed hands again. Kaufman and others in the syndicate remained financially involved, but Raskob took over as the lead developer, and Pierre Du Pont provided a substantial investment. Al Smith was installed as president of Empire State, Inc., the company erecting the structure.
However, after all the investors’ funds were pooled and combined with a MetLife mortgage, the new developers were still short the last $13.5 million needed to erect the world’s tallest building. Kaufman then put his financial acumen to work. In the fall of 1929, he created a new company, Chatham Phenix Allied Corporation (Allied), an offshoot firm owned by Chatham Phenix’s investment banking arm. Allied invested in bonds issued by the Empire State, Inc. And to raise the cash needed to buy the bonds, Allied took out a loan from Chatham Phenix Bank. (These insider dealings would later cause Kaufman great headaches when Federal officials began investigating the tangled webs of investment banking that caused the financial sector to collapse after the stock market crash.)
The Great Depression
The Empire State Building officially opened on May 1, 1931, when, ironically enough, Herbert Hoover lit up the building with a switch at the White House, at precisely 11:30 a.m. But unfortunately for its investors, including Kaufman, the deepening depression was preventing them from realizing their skyscraper dreams. In fact, throughout the 1930s, the risk of default — and the total loss of their investment — was a haunting specter, and Du Pont, Kaufman, and Raskob repeatedly provided more capital to keep their building out of bankruptcy.
Plaque in the Lobby of the Empire State Building. Photo by Thomas Redstone.
For Kaufman, in particular, his giant investment in the Empire State Building proved to be a disaster. In 1931 he lost the Chatham Phenix Bank when it was subsumed by Manufacturers Trust Company, and his name disappeared from the annals of Wall Street. His stake in the Empire State Building was also considerably pared down, but he did manage to retain some ownership of the great skyscraper that he helped to build.
Were They Fools?
Because the Empire State Building opened in the early stages of the worst economic downturn of the 20th century, the conventional wisdom is that Kaufman, Raskob, and Du Pont were ego-driven, over-ambitious fools. They threw their money into a losing proposition, it is said, because they wanted to be big machers and were eager to beat Chrysler and collect the “world’s tallest building” trophy.
But the truth is a bit more nuanced. Yes, the Great Depression created the “Empty State Building,” but the severity and length of that downturn were much more severe than anyone could have predicted. And a review of the financials of the Empire State Building in the fall of 1929 shows that they were within reasonable expectations for real estate developments of the time.
Ultimately, only Pierre Du Pont lived to see the Empire State Building fulfill its promise. In 1951, the Empire State Building was sold for the highest price ever achieved by an office building in New York City. Today, it remains the pride of New York, and each year, some four million people ride its elevators to see the Manhattan skyline from its famous observatory. History has shown that Raskob and Kaufman’s apparent delusions of grandeur were, in fact, a grand vision that continues to enrich the city.
Jason M. Barr is a professor of economics at Rutgers University-Newark. He is the author of Building the Skyline: The Birth and Growth of Manhattan’s Skyscrapers (2016) and Cities in the Sky: The Quest to Build the World's Tallest Skyscrapers (2024).
Ann Berman is a writer and author who has contributed to the Wall Street Journal, Town & Country, Forbes, Architectural Digest, Art & Auction, and many other periodicals. She is the author of a new book entitled Louis Graveraet Kaufman: The Fabulous Michigan Gatsby Who Conquered Wall Street, Took Over General Motors, and Built the Tallest Building in the World, Wayne State University Press (June 2025).