Skip to content

PIP-209: Compile Python client wrapper #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 30, 2022
Merged

Conversation

merlimat
Copy link
Contributor

No description provided.

Copy link
Contributor

@BewareMyPower BewareMyPower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still needs to import https://github.com/apache/pulsar-client-cpp as the submodule.

If the new feature is ready in C++ client, we don't have to wait a new C++ client release. It would also be better for those want to build Python library from source.

@BewareMyPower
Copy link
Contributor

In addition, some tests in pulsar_test.py might already rely on the unreleased C++ APIs as you said here.

@Demogorgon314
Copy link
Member

I think we still needs to import https://github.com/apache/pulsar-client-cpp as the submodule.

+1

@BewareMyPower
Copy link
Contributor

I'm going to work for the submodule way. The workflow (for CI in future) is:

  1. Clone the pulsar-client-cpp submodule.
  2. Add a build-client-cpp.sh to build the static library in pulsar-client-cpp.
  3. Modify the CMakeLists.txt to find the C++ static library, instead of the previous pulsarStatic target.

Regarding the dependency on lib/*.h, we can remove them later.

@shibd
Copy link
Member

shibd commented Sep 29, 2022

I think we still needs to import https://github.com/apache/pulsar-client-cpp as the submodule.

If the new feature is ready in C++ client, we don't have to wait a new C++ client release. It would also be better for those want to build Python library from source.

Will this cause the python version to be unstable? Are all versions of python built on top of unreleased versions of c++?

@BewareMyPower
Copy link
Contributor

@shibd No. It's a submodule, we need an extra PR to update the commit of the submodule if the upstream C++ client has introduced a new feature.

See https://github.com/grpc/grpc/tree/master/third_party for example.

@shibd
Copy link
Member

shibd commented Sep 29, 2022

Suddenly curious, will we print log the version that depends on C++ at startup?

Case Log:

pulsar-client-python: x.x.x
depends pulsar-client-cpp: x.x.x  or master@1d5321f

@BewareMyPower
Copy link
Contributor

will we print log the version that depends on C++ at startup?

I think it's unnecessary. Just like whether would you print the version of all dependencies in Java.

###############################################################################
if [ ! -f apache-pulsar-${PULSAR_VERSION}-src/.done ]; then
echo "Building Pulsar C++ client - ${PULSAR_VERSION}"
curl -O -L https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_VERSION}/apache-pulsar-${PULSAR_VERSION}-src.tar.gz

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably add some validation of the integrity of the archive

and also provide a way to pass a custom .tar.gz, in order to ease development/testing with a different version
maybe making the full URL configurable will help

we can do these improvements as a follow up patch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we could be checking the expected hash of the archive

Copy link
Contributor

@BewareMyPower BewareMyPower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR involved code changes, could you copy the .clang-format file from the pulsar-client-cpp repo?

CMakeLists.txt Outdated
Comment on lines 97 to 195
target_link_libraries(_pulsar -Wl,-all_load pulsarStatic ${PYTHON_WRAPPER_LIBS} ${COMMON_LIBS} ${ICU_LIBS})
target_link_libraries(_pulsar -Wl,-all_load ${PYTHON_WRAPPER_LIBS} ${COMMON_LIBS} ${ICU_LIBS})
else ()
if (NOT MSVC)
set (CMAKE_SHARED_LINKER_FLAGS " -static-libgcc -static-libstdc++")
endif()
target_link_libraries(_pulsar pulsarStatic ${PYTHON_WRAPPER_LIBS} ${COMMON_LIBS})
target_link_libraries(_pulsar ${PYTHON_WRAPPER_LIBS} ${COMMON_LIBS})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COMMON_LIBS can be removed because it's originally defined in CMakeLists.txt of pulsar-client-cpp. ICU_LIBS can be removed as well. (I never found a reference)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@merlimat merlimat merged commit 9f6fc69 into apache:main Sep 30, 2022
@merlimat merlimat deleted the build branch September 30, 2022 19:35
@BewareMyPower BewareMyPower added this to the 3.0.0 milestone Dec 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants